Computer-implemented training method, computer-implemented prediction method, computer program, computer-readable medium and device

Abstract

The invention relates to a computer-implemented training method for training machine learning models, comprising A1) a learning data collection (L, LA, LB, LC) with learning data sets (LDS) is provided, wherein each learning data set (LDS) comprises a measurement data set (MDS) with measurement data entries (MDE) and a target variable (T), A2) a grouping of the measurement data entries (MDE) of the measurement data sets (MDS) of the at least one learning data collection (L, LA, LB, LC) into multiple groups (G1, G2), A3) for each group (G1, G2), its own machine learning model is trained and tested using a resampling method, and at least one testing process with another portion of the group sub measurement data sets belonging to the respective group (G1, G2), and wherein the testing process provides predictions for different groups, group-specific predictions (P1, P2, AP1, AP2, BP1, BP2), A4) with the group-specific predictions (P1, P2, AP1, AP2, BP1, BP2) and target variables (T), at least one other machine learning model, cross-group meta learning model (MM, MA+B, MA_B), is trained, such that it can provide a cross-group prediction (P) from multiple group-specific predictions (P1, P2, AP1, AP2, BP1, BP2) associated with different groups (G1, G2). The invention also relates to a computer-implemented prediction method for predicting a characteristic using machine learning models, a computer program, a computer-readable medium and a device.

Description

[0001] Description

[0002] Computer-implemented training method, computer-implemented prediction method, computer program, computer-readable medium and device

[0003] The invention relates to a computer-implemented training method for training machine learning models, in which

[0004] A1) at least one learning data collection with a plurality of learning data sets is provided, wherein each learning data set comprises a measurement data set with a plurality of measurement data entries and a target variable assigned to the measurement data set, preferably wherein all learning data sets have the same structure.

[0005] The invention further relates to a computer-implemented prediction method for predicting a property using machine learning models, a computer program, a computer-readable medium and a device.

[0006] Artificial intelligence and machine learning (ML) are becoming important tools in the analysis of data, such as image data [Myszczynska et al. 2022]. For example, human brain image data obtained by magnetic resonance imaging (MRI) is a mere example. However, various challenges exist that complicate the application of ML methods. A major challenge, especially in the clinical field, is the availability of comparatively little data at the respective measurement site, such as a hospital, where MRI scans are taken. Since large amounts of training data are usually required to build accurate ML models, a common approach is to increase the data volume by combining data obtained from different measurement sites. However, multi-site data pooling also presents challenges.

[0007] On the one hand, harmonization is necessary because the data from different locations are heterogeneous, particularly because of systematic differences between the measurement setups used, such as scanners in the case of MRI images, as well as the data collection parameters [Chen 2021, Mali et al.]. Pooling therefore requires additional processing, which is achieved by building data harmonization models.

[0008] Furthermore, even if standard precautions, such as image pseudonymization, are observed, sharing sensitive data across sites can lead to a reduction in patient privacy, as fingerprint-like properties of the data can be used for re-identification [Finn 2015, Larabi 2021]. Currently, various methods, such as defacing MRI images, are used to increase patient privacy. Federated Learning (FL) is used to increase data protection [Kaissis et al. 2020].

[0009] The existing solutions have various limitations and disadvantages.

[0010] With cross-site data pooling, harmonization models typically require some prototype data from the respective measurement site for training. This limits applicability to new measurement sites for which no prototype data is available. Harmonization can also be problematic in terms of data protection, especially when raw data is merged. Decentralized learning, such as federated learning, can lead to models with lower accuracy. Another disadvantage arises when uninterpretable models, such as methods based on deep learning, are used. The lack of interpretability makes their use in critical decision-making situations, such as those often found in clinical settings, difficult [Hedderich & Eickhoff 2021].

[0011] Based on this, it is an object of the present invention to create a possibility to obtain particularly accurate predictions while at the same time maintaining a particularly high degree of privacy or data protection, even by using data from different measuring locations.

[0012] In a computer-implemented training method of the type mentioned above, this is solved by

[0013] A2) a grouping of at least some of the measurement data entries of the measurement data sets of the at least one learning data collection into several groups takes place, whereby group sub-measurement data sets are obtained, wherein the group sub-measurement data sets of a group each comprise mutually corresponding measurement data entries of different measurement data sets,

[0014] A3) for each group, a separate machine learning model is trained and tested using a resampling method, resampling-based group learning model, wherein the resampling-based method includes at least one training with a portion of the group sub-measurement data sets belonging to the respective group and the associated target variables, and at least one testing with another portion of the group sub-measurement data sets belonging to the respective group, and wherein the at least one testing provides predictions for different groups, group-specific predictions,

[0015] A4) at least one further machine learning model, cross-group meta-learning model, is trained with the group-specific predictions and target variables of the at least one learning data collection, in particular without using a resampling method, so that it can provide a cross-group prediction from several group-specific predictions assigned to different groups,

[0016] A5) optionally, for each group, another machine learning model is trained with group sub-measurement data sets belonging to the respective group (and associated target variables, in particular without using a resampling method), prediction group learning model, whereby the training is preferably carried out with all group sub-measurement data sets and associated target variables belonging to the respective group.

[0017] The learning models trained according to the invention can be used, in particular, to make at least one prediction for at least one new measurement data set for which no target variable is available. Purely by way of example, MRI data of a patient's brain are available, and group-specific, in particular, parcel-specific brain ages and a cross-parcel brain age are predicted for this patient. Further possible application examples for the invention can be found in genetics, medical patient data, natural language processing, and computer vision.

[0018] It should be emphasized that the steps of the training method according to the invention do not necessarily have to be performed in the aforementioned order. In particular, the training of the non-resampling-based group learning models according to step A5) can also be performed earlier, e.g., between steps A2) and A3) or simultaneously with step A3).

[0019] The invention therefore also relates to a computer-implemented prediction method for predicting a property using machine learning models obtained by carrying out the training method according to the invention, in which

[0020] B1) a measurement data set with several measurement data entries is provided,

[0021] B2) predictive group learning models trained according to step A5 and / or resampling-based group learning models trained according to step A2 and at least one cross-group meta-learning model trained according to step A4, in particular without using a resampling method, are provided,

[0022] B3) the measurement data entries of the measurement data set provided in step B1 are grouped analogously to step A2, whereby a group sub-measurement data set is obtained for each group,

[0023] B4) the group sub-measurement data sets are each fed to the corresponding prediction group learning models or the corresponding resampling-based group learning models and a group-specific prediction is obtained as output from each of these,

[0024] B5) the group-specific predictions are fed to at least one cross-group meta-learning model, which is trained in particular without using a resampling method, and from which a cross-group prediction is obtained as output.

[0025] It should be emphasized that the steps of the inventive prediction method do not necessarily have to be performed in the above-mentioned order. For example, the learning models can also be provided after the grouping.

[0026] In other words, the present invention provides a "divide and conquer" approach in which multiple levels of machine learning models are trained. During the prediction process using the trained models, multiple levels are also passed through. The outputs of each level are used—both in training and prediction—as inputs for the next, "higher" level.

[0027] According to the invention, the training data is divided into groups, and, particularly in the first, "lowest" level (Level 0), a machine learning model is introduced for each group and trained and tested using a resampling-based method—in other words, group-specific learning models or group learning models. These are trained and tested with the training data using a resampling-based method.

[0028] Within the scope of the grouping, different parts or sections of the measurement data sets are expediently assigned to different groups. Purely as an example, a subdivision into two or more groups takes place, with a part of the measurement data entries of the measurement data sets being assigned to one group and the other part(s) of the measurement data entries being assigned to the other group(s). The grouping is such that corresponding parts or areas of the measurement data sets, i.e. corresponding measurement data entries, are each assigned to the same group. By way of example, a first group comprises the first five measurement data entries of the measurement data sets, a further group the next seven measurement data entries of all measurement data sets, and so on. If the measurement data sets are given by image data or comprise such data, a group in particular includes the same image section of each measurement data set, i.e. each image orevery recording.

[0029] The groups can be mutually exclusive, in other words disjoint, or they can overlap, i.e. they can contain common measurement data entries. The group sub-measurement data records of different groups should be different from one another. The groups can be flat or organized in a structure, e.g. a hierarchy. It is possible for all measurement data entries to be assigned to one or more groups, or for this to apply to only some of the measurement data entries. Furthermore, it is possible for only one measurement data entry or several measurement data entries to be assigned to a group. For example, one or more groups, or each group, can consist of exactly one measurement data entry.

[0030] For example, a grouping predefined by a user or a grouping of the measurement data sets derived from the respective learning data collection, in particular the structure of the measurement data sets, preferably obtained by clustering, can be carried out.

[0031] The resampling method, as is well known, involves dividing the data one or more times into a training and test section; one can also speak of training and test sets. The sub-measurement data sets along with the associated target variables are then used from one section, the training section, to train the respective group learning model so that, in the trained state, it can predict (further) target variables. In this way, machine learning models are trained for each group, using measurement data sets assigned to the respective group. Each sub-measurement data set is part of a measurement data set and thus assigned to or belongs to it. Each measurement data set is assigned a target variable, which can accordingly also be assigned to the sub-measurement data set or considered to belong to the sub-measurement data set.

[0032] The sub-measurement datasets belonging to the other part, the test part, of the respective group are then fed (without the corresponding target variables) into the respective group learning model in order to obtain predictions from it. This is also referred to as "out-of-sample" predictions. Training with one part and testing with another part of the data can be carried out multiple times in a conventional manner within the framework of resampling-based modeling. Purely as an example, the measurement datasets can be divided into two parts of equal or different size. In a first pass, one part is trained and the other is predicted, and in a second pass, the datasets are swapped, i.e., the other part is trained and the one part is tested. In this way, one can obtain a number of "out-of-sample" predictions that corresponds to the total number of measurement datasets, although this is by no means mandatory, but merely an option.Since the resampling-based modeling is carried out for each group, group-specific predictions are obtained.

[0033] At least one meta-model can then be trained using the group-specific predictions and the target variables of the training dataset. After training, the at least one meta-model is capable of delivering a common, cross-group prediction as output based on several group-specific predictions as input. Optionally, in step A5, a further machine learning model is trained for each group using group sub-measurement datasets belonging to the respective group and associated target variables, expediently without using a resampling method. These models are also referred to herein as prediction group learning models, as they can be used primarily for the prediction method according to the invention. A learning model without using a resampling method also includes learning models that were derived from or combined with previously trained resampling-based learning models.Examples include ensemble models from models that were previously trained based on resampling, e.g., by k-fold cross-validation.

[0034] At prediction time, a new instance sequentially passes through all levels of the stack—that is, the trained machine learning models at all levels—and obtains predictions for each level. The final prediction from the last, highest level and the intermediate predictions from the previous level(s) can be used for interpretation and decision-making.

[0035] In principle, various types of predictions can be made within the framework of the training and prediction method according to the invention, for example predicted values ​​or prediction probabilities or confidence. The prediction target variables can accordingly be or can include, for example, prediction values, prediction probabilities or confidences. It should be noted that the term target variable can also be used as an alternative to the term target variable. Since the models in each level predict the result or target, the output of each level is more focused on the target than on any other information contained in the raw data, in particular the measurement data sets. In other words, by outputting target predictions or estimates, each level creates a more abstract representation of the data and thus reduces private information.This makes it possible to ensure a high degree of privacy, especially when sharing or merging data across measurement sites, which represents a key advantage of the invention. The invention allows for less private data to be shared, while simultaneously allowing more accurate models to be obtained using data from multiple locations (“cross-site”).

[0036] Purely examples of resampling methods that can be used within the scope of the invention include bootstrapping and cross-validation, such as leave-one-out cross-validation or k-fold cross-validation. Of course, a combination of different resampling methods can also be used, both within a single level and across different levels.

[0037] The training of machine learning models within the scope of the present invention may further comprise the tuning of hyperparameters, whereby this applies to all learning models at all levels.

[0038] It has proven particularly suitable if two, three or four levels or layers of machine learning models are provided within the scope of the present invention. One can also speak of a stack with two, three or four levels. However, it is by no means excluded that more levels are used. In particular, a number of levels or layers of the stack can be provided that is adapted to the grouping. Levels can also be organized hierarchically. Purely as an example, the hierarchical structuring of brain MRI data based on the brain parcellation scheme of Yeo et al. (2011) is mentioned. Here, it would be possible to first use the voxels in Level 1 based on their brain parcellations as a grouping to model the target variable. In Level 2, the predictions per brain parcel could be used to model the target variable as a grouping based on its brain networks defined by Yeo et al.This is a hierarchical structure in which each network consists of multiple parcels, each consisting of multiple voxels. In this example, not all voxels would be used as input, as this brain parcellation scheme considers only the cerebral cortex. All unused voxels could be ignored in the training process or, by adding other parcellation schemes, used for further hierarchical or non-hierarchical modeling. This means that in a Level 4, the predictions can now be grouped and used based on different parcellation schemes. This includes either different networks or brain parcellations, or a mixture of both. This is just one example of the possibilities that are conceivable within the scope of the present invention.Other combinations of hierarchical or non-hierarchical, overlapping or non-overlapping groupings are also possible.

[0039] The machine learning models used within the scope of the present invention can be of any type. In particular, a user can freely choose which model(s) to select for the various groups and levels. The models can, in particular, comprise or be provided by one or more machine learning algorithms. Decision trees and support vector machines are purely examples of machine learning algorithms. If inherently interpretable models are selected, the overall solution becomes interpretable as such.

[0040] In one embodiment of the training method according to the invention, it is provided that the different learning data sets belong to different patients and the learning data sets comprise medically relevant measurement data sets, in particular measurement data sets obtained by means of medical diagnostic methods, and the target variables relate to or are given by characteristics of the patients, in particular an age and / or a medical condition of the patients.

[0041] The measurement data sets can, for example, be provided by or include images, in particular MRI images, of human brains. The grouping provided according to the invention can then be carried out in parcels, so that each group corresponds to a specific parcel with specific, expediently connected voxels. The term voxel is composed of "volumetric" and "pixel" and corresponds, in a known manner, to the 3D equivalent of a pixel. The measurement data entries can then, in particular, be voxel-wise measurements of MRI data, such as the gray matter volume, with each group containing the voxels of a specific brain parcel.

[0042] Particularly in the specific case of brain MRI, the invention can be used to exploit known regularities in the data to initially make location-based predictions, which are then followed by final, brain-wide predictions—in other words, predictions associated with the entire brain. For example, one can imagine an age or gender prediction that can serve as model cases for regression or classification. These model cases offer clinical applicability, are well-established, are known to offer high accuracy, and their results, including the significance of the features, are easy to verify. For example, it is expected that the volume of gray matter decreases with increasing age, so a negative feature weighting can be assumed. Within the scope of the invention, the lowest level of the stack will build predictive models using a priori defined groupings of brain regions.Following the training phase, the trained group-specific models and the meta-model(s) can be used to make predictions for a provided measurement dataset, for example, an MRI scan of a patient.

[0043] In the case of MRI images of patients, such as their brains, the target variables can be the patient's age or disease status, for example. Within the framework of the prediction procedure, a patient's age or disease status can be predicted accordingly.

[0044] A further embodiment is characterized in that the different learning data sets belong to different individuals, and the measurement data sets are provided by or comprise images of at least part of the face and / or at least part of the body of the individuals, and the target variables relate to or are defined by characteristics of the individuals. The measurement data entries can then each correspond to or be assigned to a pixel. Furthermore, it can be provided that the characteristics of the individuals do not relate to a medical condition of the individuals or are not defined by a medical condition of the individuals.

[0045] In other words, training can also be performed using images of faces or parts of faces and / or bodies or body parts of people, as well as characteristics of the people, such as age and / or gender. Following the training phase, the trained group-specific models and the meta-model(s) can be used to make predictions for a provided measurement dataset, such as a picture of a person's face, e.g., to predict the person's age and / or gender.

[0046] With regard to the grouping provided for in the invention, in this case, for example, it may apply that a grouping is carried out according to certain areas or parts of the face (e.g. nose, eyes, ...) and / or body.

[0047] It is also possible for the different learning data sets to belong to different sections of the Earth's surface and for the measurement data sets to be given by or include image recordings, in particular satellite images, of the Earth's surface sections, and for the target variables to relate to or be given by properties of the Earth's surface sections, in particular the presence of certain elements, preferably the presence of fields and / or rivers and / or lakes.

[0048] In other words, training can be carried out using satellite images of the Earth's surface and specific landscape features belonging to the respective section shown, such as the existence of fields, lakes, rivers, etc. Following the training phase, the trained group-specific models and the meta-model(s) can be used to make predictions for a new satellite image with unknown features, for example, to predict whether it shows rivers, lakes and / or fields, to name just a few examples of possible features. With regard to the grouping provided according to the invention, in this case, in particular, grouping can take place according to specific areas or parts of the image recordings. For example, segmentation (image segmentation) can be carried out, within the framework of which regions or areas that are preferably related in terms of content and that include neighboring pixels that meet a specific homogeneity criterion are determined.The resulting segments can then each represent a group or correspond to one.

[0049] As a further example, the different learning data sets belong to different individuals, and the measurement data sets are provided by or include information about the individuals' user behavior, in particular on at least one website, and the target variables relate to or are provided by information about actions performed by the individuals, in particular purchases made by the individuals. These are expediently actions performed on the at least one website, such as purchases of products and / or services offered on the at least one website.

[0050] The information about people's user behavior included in the measurement data sets can, for example, include information about how long people spent on a website, to which area(s) of the website they moved the mouse pointer, and in particular, how long the mouse pointer remained in each area, and / or which areas they clicked with the mouse pointer. In other words, training can be carried out using information about user behavior, particularly on the internet, and specific associated user actions, such as purchases.

[0051] Following the training phase, the trained group-specific models and the meta-model(s) can be used to make predictions about actions or probabilities of actions, such as purchase probabilities for certain products and / or services, based on data on the user behavior of at least one other person.

[0052] With regard to grouping, it may then apply, for example, that different areas of the at least one website form different groups or correspond to such, in other words a grouping into different website areas or sections takes place.

[0053] A further embodiment is characterized in that the various learning data sets belong to different DNA sequences and / or protein sequences and / or gene expressions, and the measurement data sets are provided by or comprise information about the DNA sequences and / or protein sequences and / or gene expressions. This information then expediently concerns information concerning the structure of the DNA sequences or protein sequences or gene expressions. The information can comprise sequence-based features of DNA sequences and / or protein sequences or gene expression data. The target variables then expediently concern (other) features of the DNA sequences and / or protein sequences and / or gene expression data, in particular binding sites and / or protein-protein interactions and / or solvent properties thereof, or are provided thereby.Following the training phase, the trained group-specific models and the meta-model(s) can then be used, for example, to predict binding sites or protein-protein interactions or solvent properties based on information for a new DNA or protein sequence or gene expression data, in particular based on an associated measurement data set.

[0054] Regarding grouping, it is possible, for example, to define groups using domain knowledge, such as known gene regions or interaction networks.

[0055] The measurement data for training can come from one or more measurement locations. In the latter case, separate group learning models—in other words, measurement location-specific group learning models—can be introduced for each measurement location.

[0056] In a preferred development of the training method according to the invention, it is therefore provided that in step A1, learning data collections are provided from and in particular at different measurement locations, wherein the learning data collections each comprise a plurality of learning data sets with measurement data entries and associated target variables.

[0057] If cross-site data is to be used for training, this usually poses a particularly significant problem with regard to privacy and data protection, as the data from the various measurement sites must be shared—in other words, handed over—so that it can be used at another measurement site or even a third party to train learning models. This is especially true when it comes to data of medical relevance and / or other particularly sensitive data. Decentralized learning does offer the possibility of greater privacy, as the data does not have to be centrally consolidated but can remain at the respective measurement site. However, as already noted, decentralized learning generally leads to less accurate models.

[0058] The invention offers a major advantage here, as sharing or merging data is possible at various of the provided levels, with privacy and data protection being greater the higher the level at which the sharing or merging occurs. However, even the predictions at the lowest level, i.e., the group-specific or group-wise predictions, contain no or at least only limited private information, since their information content relates only to the prediction, in other words, the predicted target. Accordingly, this data can be shared across measurement locations to train ML models while still maintaining privacy.

[0059] Cross-site models can be introduced or maintained in various ways within the scope of the invention. For example, intermediate fusion can be performed, in which predictions from the lowest-level models are shared across measurement sites to train an ML model with the pooled data. It is also possible to perform "late fusion," in which predictions from the next higher level are shared first, resulting in a weighted aggregation ensemble model. In all cases in which the predictions of the models are merged, whether at the lowest or higher levels, harmonization effects are achieved and privacy is increased.An embodiment of the training method according to the invention is characterized in that in step A3, for each measurement location, separate resampling-based measurement location-specific group learning models are trained and tested with the learning data sets belonging to the respective measurement location, wherein the resampling method includes in each case that at least one training is carried out with a part of the measurement location-specific group sub-measurement data sets and associated measurement location-specific target variables, and at least one testing is carried out with another part of the measurement location-specific group sub-measurement data sets, and wherein the at least one testing for the respective measurement location provides predictions for different groups, measurement location- and group-specific predictions.

[0060] Sharing the learning data sets, in other words, raw data, is then unnecessary. The data from each measurement location can be processed independently of the data from other measurement locations, with the inventive grouping and resampling-based training and testing to obtain group-specific predictions being performed site by site. As a result, group-specific predictions are available for each individual measurement location, which are also referred to herein as site- and group-specific predictions.

[0061] It can then further be provided that, in step A5, separate location-specific prediction group learning models are trained for each measurement location, in particular without using a resampling method, whereby the training is preferably carried out with all location-specific group sub-measurement data sets and associated location-specific target variables. Location-specific prediction group learning models obtained in this way can be used, in particular, for subsequent prediction. In step A4, the location- and group-specific predictions and associated location-specific target variables from different measurement locations can then be combined, thus training a consolidated meta-learning model across all measurement locations and groups, in particular without using a resampling method. This scenario corresponds to the intermediate data fusion already mentioned above.

[0062] Furthermore, it can be provided that in step A4, a resampling-based, site-specific, cross-group meta-learning model is trained and tested for each measurement location, wherein the resampling method includes at least one training with a portion of the site- and group-specific predictions and the associated site-specific target variables of the learning data collections, and at least one testing with another portion of the site- and group-specific predictions and the associated site-specific target variables of the learning data collections, and wherein the at least one testing provides site-specific cross-group predictions.

[0063] This has proven particularly useful when no intermediate data fusion occurs and the site- and group-specific forecasts from different sites are not shared or merged.

[0064] Using the site-specific, cross-group predictions and associated site-specific target variables originating from multiple measurement locations, a cross-site meta-learning model can then be trained, preferably without using a resampling method, so that it can deliver a site- and group-wide prediction from multiple site-specific, cross-group predictions. Further preferably, in particular in step A4, a site-specific, cross-group meta-learning model is then trained for each measurement location, preferably without using a resampling method, wherein the training is carried out using the site- and group-specific predictions and the associated site-specific target variables of the respective learning data collection, preferably using all site- and group-specific predictions and associated site-specific target variables.The non-resampling-based site-specific cross-group meta-learning models obtained in this way can be used in particular for subsequent prediction.

[0065] It can also be provided that an additional learning data collection is provided from an additional measuring location different from the measuring locations, and a grouping of the measurement data entries of the measurement data sets of the additional learning data collection into several groups as provided in step A2, whereby additional group sub-measurement data sets are obtained, the additional group sub-measurement data sets and the target variables of the additional learning data collection are each fed to the measurement location-specific prediction group learning models belonging to the measurement locations, in particular trained without using a resampling method, and from these measurement location- and group-specific predictions are obtained,the measurement location- and group-specific predictions are fed into the measurement location-specific cross-group meta-learning models belonging to the measurement locations, which are trained in particular without the use of a resampling method, and from these, measurement location-specific cross-group predictions are obtained; an additional cross-measurement location meta-learning model is trained with the measurement location-specific cross-group predictions and the target variables of the additional learning data collection, preferably without the use of a resampling method.

[0066] In other words, it is possible to use trained learning models belonging to different measurement locations to make predictions for data from yet another measurement location. Purely as an example, learning models are trained using data from one measurement location A, and learning models are trained using data from one measurement location B, thus obtaining measurement location-specific learning models for A and B. Data from a third measurement location C is then provided, and the data from C is fed into both the learning models belonging to measurement location A and the learning models belonging to measurement location B, and predictions are obtained. It should be emphasized that the scenario with measurement locations A, B, and C is purely exemplary, and a different number of measurement locations is of course also possible.

[0067] In an advantageous development of the training method according to the invention, it is further provided that the trained site-specific, cross-group meta-learning model of at least one measurement site is fed with site- and group-specific predictions from at least one other measurement site, along with associated target variables, thereby obtaining site-specific, cross-group cross-predictions. Preferably, all site- and group-specific predictions and associated target variables of the (respective) other measurement site are fed.

[0068] The cross-measurement location meta-learning model can then be trained with the measurement location-specific cross-group predictions and the measurement location-specific cross-group cross-predictions. In the prediction method according to the invention, an advantageous development provides that, in step B2, measurement location-specific prediction group learning models of different locations obtained by implementing the training method according to the invention and / or resampling-based measurement location-specific group learning models of different locations obtained by implementing the training method according to the invention, and a consolidated cross-measurement location- and cross-group meta-learning model obtained by implementing the training method according to the invention, in particular without using a resampling method, are provided.the group sub-measurement data sets obtained in step B3 are fed to the measurement location-specific prediction group learning models or the resampling-based measurement location-specific group learning models of different measurement locations in step B4, and measurement location- and group-specific predictions are obtained; the predictions of different measurement locations belonging to a group are combined with each other using a statistical method, in particular averaging, so that a group-specific prediction is obtained for each group; the group-specific predictions are fed to the merging meta-learning model spanning measurement locations and groups, and a prediction spanning measurement locations and groups is obtained from this model.

[0069] This approach has proven particularly useful in cases where intermediate data fusion was performed during training, in which predictions from the lowest-level models were combined or merged. Alternatively or additionally, it can be provided that

[0070] - in step B2, measurement location-specific prediction group learning models of different locations obtained by carrying out the training method according to the invention and / or resampling-based measurement location-specific group learning models of different locations obtained by carrying out the training method according to the invention and a consolidated meta-learning model across measurement locations and groups, which was obtained by carrying out the training method according to the invention, in particular without using a resampling method, are provided,

[0071] - the group sub-measurement data sets obtained in step B3 are fed to the site-specific prediction group learning models or the resampling-based site-specific group learning models of the different measurement locations in step B4 and site- and group-specific predictions are obtained,

[0072] - the measurement location- and group-specific predictions are fed into the consolidated meta-learning model across measurement locations and groups, thus obtaining measurement location-specific cross-group predictions for each measurement location,

[0073] - the site-specific cross-group forecasts are combined using a statistical method, in particular averaging, so that a site- and group-wide forecast is obtained.

[0074] This has proven to be a further suitable prediction variant, especially in the case where an intermediate data fusion was performed during training. A further advantageous embodiment of the prediction method according to the invention is further characterized in that

[0075] - in step B2, measurement location-specific prediction group learning models of different locations obtained by carrying out the training method according to the invention, and / or resampling-based measurement location-specific group learning models of different locations obtained by carrying out the training method according to the invention, and measurement location-specific cross-group meta-learning models obtained by carrying out the training method according to the invention, in particular without using a resampling method, and a cross-measurement location meta-learning model obtained by carrying out the training method according to the invention, in particular without using a resampling method, and / or an additional cross-measurement location meta-learning model obtained by carrying out the training method according to the invention, in particular without using a resampling method, are provided,

[0076] - the group sub-measurement data sets obtained in step B3 are fed to the measurement site-specific prediction group learning models or the resampling-based measurement site-specific group learning models of the different measurement sites in step B4 and measurement site- and group-specific predictions are obtained,

[0077] - the measurement location- and group-specific predictions are fed to the respective corresponding measurement location-specific cross-group meta-learning model, thus obtaining measurement location-specific cross-group predictions (for each measurement location,

[0078] - the site-specific cross-group predictions are fed into the cross-site meta-learning model or the additional cross-site meta-learning model, from which a site- and group-wide prediction is obtained.

[0079] This embodiment is particularly suitable for the case where late data fusion was performed during training, in which predictions of the model(s) of a higher than the lowest level were combined or merged.

[0080] A further subject matter of the present invention is a computer program comprising instructions which, when executed on at least one computer, cause the at least one computer to carry out the steps of the training method according to the invention and / or the steps of the prediction method according to the invention.

[0081] The invention also relates to a computer-readable medium comprising instructions which, when executed on at least one computer, cause the at least one computer to carry out the steps of the training method according to the invention and / or the steps of the prediction method according to the invention.

[0082] It is also possible for at least some of the steps of the training method according to the invention and / or at least some of the steps of the prediction method according to the invention to be performed in a cloud. This includes, in particular, the use of externally, remotely, or distributed services via the Internet (cloud computing), which includes "Infrastructure as a Service" (IaaS), "Platform as a Service" (PaaS), and "Software as a Service" (SaaS).

[0083] Finally, the subject of the invention is a device for carrying out the training method according to the invention and / or the prediction method according to the invention, comprising a computer with a data memory on which the computer program according to the invention is stored.

[0084] In a preferred embodiment, the device according to the invention comprises an MRI scanner device. This device is designed and configured to create MRI images of a patient. The device according to the invention has proven particularly useful for locally executing the training method and / or prediction method according to the invention.

[0085] With regard to the embodiments of the invention, reference is also made to the subclaims and to the following description of several embodiments with reference to the accompanying drawings.

[0086] The drawing shows in a purely schematic representation:

[0087] Figure 1 shows a first embodiment of the computer-implemented training method according to the invention for the case where the learning data sets originate from a measurement location, and a first embodiment of the computer-implemented prediction method according to the invention which uses trained models obtained with the training method according to this figure;

[0088] Figure 2 MRI images of human brains of the learning data collection from Figure 1 along with a purely schematic representation for grouping into brain parcels;

[0089] Figure 3 shows a second embodiment of the computer-implemented training method according to the invention for the case where the learning data sets originate from two different measurement locations, and a second embodiment of the computer-implemented prediction method according to the invention which uses trained models obtained with the training method according to this figure;

[0090] Figure 4 shows a third embodiment of the computer-implemented prediction method according to the invention, which also uses trained models obtained with the training method according to Figure 3;

[0091] Figure 5 shows a third embodiment of the computer-implemented training method according to the invention for the case where the learning data sets originate from two different measurement locations, and a fourth embodiment of the computer-implemented prediction method according to the invention, which uses trained models obtained with the training method according to this figure; and

[0092] Figure 6 shows a fourth embodiment of the computer-implemented training method according to the invention for the case where the learning data sets originate from three different measurement locations, and a fifth embodiment of the computer-implemented prediction method according to the invention, which uses trained models obtained with the training method according to this figure.

[0093] In the figures, identical or corresponding elements are provided with identical reference numerals. Figure 1 shows, at the top, a purely schematic block diagram of steps of a first exemplary embodiment of the computer-implemented training method Tr according to the invention. These steps are summarized in a frame labeled Tr. Below this is another frame, labeled Vo, within which the steps of a first corresponding exemplary embodiment of the computer-implemented prediction method according to the invention are shown, which uses machine learning models MG1, MG2, MM obtained with the training method according to Figure 1. In the figure, the use is also indicated schematically by dashed arrows.

[0094] Figures 3, 5 and 6 have a structure analogous to Figure 1, each with an embodiment of a training method according to the invention at the top and an embodiment of a prediction method according to the invention at the bottom, which uses models from the respective above method.

[0095] In the training method according to Figure 1, a learning data collection L with multiple learning data sets LDS is provided in a step A1. Each learning data set LDS comprises a measurement data set MDS with multiple measurement data entries ME and a target variable T assigned to the measurement data set MDS. In the example shown here, all learning data sets LDS have the same structure, in particular the same number of measurement data entries MDE, each with an assigned target variable T. The measurement data entries ME can also be referred to as features, which is indicated in the figure by the abbreviation F.

[0096] In the example shown here, the learning data sets LDS are MRI images of human brains from various patients of different ages. This is illustrated in Figure 2, which shows four such images as an example. MRI stands for magnetic resonance imaging. The MRI images were obtained in a well-known manner using a magnetic resonance imaging system, which can be found, for example, in a hospital in which the patients were examined. The patient ages - here, for example, 42, 21, 53, and 78 years - are also given in Figure 2 for the respective image. The measurement data entries MDE are given by the voxel-wise gray matter volume (voxel-wise GMV for short).

[0097] In step A2 of the training method according to Figure 1, the measurement data entries MDE of the measurement data sets MDS of the learning data collection L are grouped Gr into several groups G1, G2. This results in group sub-measurement data sets GUM, whereby the group sub-measurement data sets GUM of a group G1, G2 each comprise mutually corresponding measurement data entries ME of different measurement data sets MDS. For reasons of clarity, this is shown in Figure 1 as an example for two groups G1, G2. It should be emphasized that grouping into a larger number of groups is of course possible and will usually occur.

[0098] In the example shown here, the brain is grouped into parcels of a brain parcellation scheme, also called a brain atlas. One parcel can be defined, for example, by the hippocampus, and further parcels by other brain regions. Another brain atlas is the Schaefer Atlas [Schaefer et al. 2018]. In Figure 2, two parcels PA1 and PA2 are shown purely schematically in the frontmost MRI image. As can be seen, the one parcel PA1 on the right in Figure 2, which corresponds to the first group G1, comprises nine neighboring voxels, and the second parcel PA2 on the left in Figure 2, which corresponds to group G2, comprises four neighboring voxels; this is also to be understood purely as an example. For the other MDS measurement data sets, in this case MRI images of other patients of a different age, the grouping is carried out in a completely analogous manner.It should be noted that, although Figure 1 shows a different number of measurement data entries (MDE) for each group G1, G2 for reasons of clarity, the principle can be derived equally here. Furthermore, it should be noted that in the purely schematic representation of the training data set L in Figure 1, each row corresponds to a patient.

[0099] Finally, it should be emphasized that, even though the method according to the invention has proven particularly suitable for MRI images, it can in principle also be applied to any other type of data. Further application examples include other medical patient data, genetic data, natural language processing, and computer vision.

[0100] For example, it is also possible for the different learning data sets LDS to belong to different individuals, and for the measurement data sets MDS to be provided by or include images of at least part of the face and / or at least part of the body of the individuals, and for the target variables T to relate to or be provided by characteristics of the individuals. In this case, grouping into different areas of the face or body can occur, for example.

[0101] As a further example, the various learning data sets LDS belong to different sections of the Earth's surface, and the measurement data sets MDS are provided by or include images, particularly satellite images, of the Earth's surface sections. The target variables T relate to or are provided by properties of the Earth's surface sections, particularly the presence of certain elements, for example, the presence of fields and / or rivers and / or lakes. It would then be possible, for example, to group the data according to specific areas or parts of the image recordings.

[0102] It is also possible for the different learning data sets LDS to belong to different individuals, and for the measurement data sets MDS to contain information about the individuals' user behavior, in particular on at least one website, and for the target variables T to relate to or be provided by information about actions performed by the individuals, in particular purchases made by the individuals. Then, for example, different areas of the at least one website can form different groups or correspond to such groups, in other words, a grouping into different website areas or sections can occur.

[0103] As a further example, the various learning data sets LDS belong to different DNA sequences and / or protein sequences and / or gene expressions, and the measurement data sets MDS are given by or comprise information about the DNA sequences and / or protein sequences and / or gene expressions, in particular by the structure of information relating to these, and the target variables T relate to or are given by characteristics of the DNA sequences and / or protein sequences and / or gene expression data, in particular binding sites and / or protein-protein interactions and / or solvent properties thereof. Then, for example, different DNA or gene regions can form or correspond to different groups.

[0104] After the inventive grouping into brain parcels—or one of the other groupings described above as examples—has been performed, in step A3, a separate machine learning model is trained and tested for each group G1, G2 using a resampling method. This is also schematically indicated in Figures 1 and 2, in Figure 1 by the arrow RsB M, where the abbreviation stands for resampling-based modeling. The model RsB MG1 is trained for group G1, and the model RsB MG2 is trained for group G2 (see Figure 2).

[0105] In principle, the user is free to choose the type of learning model to be used. The models include one or more machine learning algorithms, which can be, for example, decision trees and / or support vector machines. Inherently interpretable models are preferred.

[0106] The resampling-based method includes at least one training step using a portion of the group sub-measurement data sets GUM belonging to the respective group G1, G2 and the associated target variables T, and at least one testing step using another portion of the group sub-measurement data sets GUM belonging to the respective group G1, G2, and wherein the at least one testing step provides predictions for different groups G1, G2, group-specific predictions. In a manner known per se, the data is subdivided one or more times into a training and test portion; one can also speak of training and test sets.In the present example, the data - specifically the group sub-measurement data sets GUMs with associated target variables T belonging to the respective group G1, G2 for some patients, for example, one half of the patients, are used as training sets, and the data GUMs of the remaining patients, i.e., the second half, are used as test sets. Cross-validation, such as leave-one-out cross-validation or k-fold cross-validation, is performed. The group-wise feeding of the training set GUMs and associated target variables T to the group learning models RsB MG1, RsB MG2 is also indicated purely schematically in Figure 2 on the right.

[0107] Using the test set GLIMs, group-wise “out-of-sample” predictions P1, P2 (see Figure 1) for the patient age are then obtained. Since the group-wise approach results in separate predictions P1, P2 for each group, these are also referred to as group-specific predictions P1, P2. The process then switches; in other words, the data from the second half is used for training and the data from the first half for testing, so that “out-of-sample” predictions P1, P2 are also obtained for the first half. With this approach, group-specific predictions P1, P2 can be obtained in a number that corresponds to the number of measurement data sets MDS in the training data set L and thus to the total number of patients. However, this is by no means mandatory.

[0108] Using the group-specific predictions P1, P2 of the patient ages and the known patient ages, i.e., target variables T of the training data set L, another machine learning model, a cross-group meta-learning model (MM), is trained in step A4 without using a resampling method. One could also say that several (in this example, two) levels of machine learning models are trained, whereby the group-wise training can be referred to as LevelO training and the training of the meta-model (MM) as LevelH training.

[0109] In Figure 1, the training without use of a resampling method of the meta-model MM is indicated by an arrow ML M. Following this training, the model MM can then provide a cross-group prediction P from several group-specific predictions P1, P2 assigned to different groups G1, G2. Furthermore, in step A5, for each group G1, G2, another machine learning model is trained without use of a resampling method with group sub-measurement data sets GMU belonging to the respective group G1, G2 and associated target variables T, prediction group learning model MG1, MG2, whereby the training here takes place with all group sub-measurement data sets GMU and associated target variables T belonging to the respective group G1, G2, in other words here with the learning data set data of all patients.In Figure 1, this training is indicated purely schematically by another arrow, ML M, and to the right of it, the predictive group learning models MG1 and MG2. Training of MG1 and MG2 can, of course, also occur before training the model MM, e.g., before, after, or parallel to the resampling-based training of RsB MG1 and RsB MG2.

[0110] The obtained trained learning models MM and MG1 , MG2 can then be used to estimate the age of other patients based on their brain MRI images.

[0111] As noted above, this is shown purely schematically in Figure 1 below. It should be noted that training the prediction group learning models MG1 and MG2 is optional, since the resampling-based group learning models RsB MG1 and RsB MG2 can, in principle, also be used for prediction as an alternative. Additional use is also possible.

[0112] In the event that training was carried out with measurement data sets MDS with images of at least part of the face and / or at least part of the body of persons and associated target variables with characteristics of the persons, for example, the age and / or gender of another person can be predicted based on a new image.

[0113] In the event that training with measurement data sets MDS was carried out with image recordings, in particular satellite images, of earth surface sections and associated target variables (T) with properties of the earth surface sections, it can be predicted, for example, for a new recording of another earth surface section whether this includes at least one river and / or at least one lake and / or at least one field.

[0114] In the event that training was carried out using measurement data sets MDS with information about the user behavior of people and associated target variables T with information about actions performed by the people, a purchase probability can be predicted, for example, for a measurement data set MDS with information about the user behavior of another person.

[0115] In the event that training was carried out with measurement data sets MDS with information about DNA sequences and / or protein sequences and / or gene expressions, for example, at least one binding site or protein-protein interaction can be predicted for a measurement data set MDS.

[0116] Specifically, for the prediction, a measurement data set (MDS) with multiple measurement data entries (MDE) is provided in step B1. These can also be referred to as test sample features.

[0117] In step B2, predictive group learning models MG1, MG2 trained according to step A5 above and a cross-group meta-learning model MM trained according to step A4 above are provided. The measurement data entries MDE of the provided measurement data set MDS are grouped in step B3 analogously to step A2, whereby a group sub-measurement data set GUM is obtained for each group. Analogous to step A2 means in the present case, or in the case of training with MRI images, that the new MRI image of the patient of unknown age or the corresponding voxel-wise gray matter volumes are divided into the same brain parcels as occurs during the training process. It is understood that even if the case of MRI images of brains is discussed below as an example, the procedure can be completely analogous with regard to the other aforementioned examples.

[0118] In step B4, the group sub-measurement data sets GUM are each fed to the corresponding prediction group learning models MG1, MG2, from which a group-specific prediction P1, P2 is obtained as output. As noted, the models RsB MG1, RsB MG2 can also be used alternatively or additionally (not shown additionally in Figure 1 below for reasons of clarity). In the embodiment described here, a patient age is obtained as a prediction P1, P2 for each group, i.e., each brain parcel. The group-specific, i.e., parcel-specific ages can differ from one another.

[0119] In step B5, the group-specific predictions for the age P1 , P2 are fed to the cross-group meta-learning model MM and from this, exactly one cross-group prediction P, in other words a patient age for the whole brain, is obtained as output.

[0120] The advantages of this approach include increased prediction accuracy, improved generalizability of the model predictions, and increased interpretability / explainability at both the model level and the data point level. Interpretability at the model level means that each trained model also provides out-of-sample predictions and thus prediction accuracy. This prediction accuracy can be interpreted as a measure of the importance of the predictive power of the measurement data entries (MDE) used by this model, and thus of the groupings. Interpretability at the data point level is achieved by testing the prediction accuracies of each model for data points - including new ones - in the prediction process, and can serve as an explanation for differences in the prediction accuracy of the entire prediction process for this data point. If the prediction accuracy for models with certain measurement data entries (MDE) orIf grouping as input is different than expected, this can provide information as to why the prediction of the entire prediction process is different than expected.

[0121] The computer-implemented training method according to the invention is particularly well-suited for cases where learning data is available from or at different measurement locations. An exemplary embodiment of such a cross-site scenario is shown purely schematically in Figure 3. The following will specifically address how the method in this case differs from that shown in Figure 1. Regarding the remaining, similar aspects, reference is made to the preceding description.

[0122] In contrast to the example in Figure 1, in the cross-site scenario, several learning data collections originating from different measurement locations are provided in step A1. In the example according to Figure 3, specifically two learning data collections LA, LB originating from measurement locations A, B. It is understood that Figure 3 shows two measurement locations A, B purely as an example and for reasons of clarity, and there can of course be more. The different measurement locations A, B are provided here by different hospitals, each of which has its own MRI system, using which the measurement data sets MDS of the learning data collections LA, LB for patients of known age were obtained. Each of the two learning data collections LA, LB is similar in structure to the learning data collection L from Figure 1, so that reference is also made to the above description in this regard.

[0123] In the cross-site scenario according to Figure 3, in step A3, for each and at each measurement location A, B, separate resampling-based, site-specific group learning models are trained and tested using the learning data sets LDS belonging to the respective measurement location A, B. This is again indicated in the figure by arrows labeled RsB M. Here, too, the resampling method includes at least one training with a portion of the site-specific group sub-measurement data sets GUM belonging to the respective measurement location A, B and the associated site-specific target variables T, and at least one testing with another portion of the site-specific group sub-measurement data sets GUM. The testing then provides - for the respective measurement location A, B - predictions for different groups, in other words, site- and group-specific predictions AP1, AP2, BP1, BP2.In Figure 3, for better differentiation, the hatching is tilted to the left for location A and to the right for location B. The forecasts AP1 and BP1 specific to location A and B, respectively, and group G1 are the ones hatched to the left and right, respectively, and labeled P1. The same applies to group G2.

[0124] Just as in the example shown in Figure 1, cross-validation, such as leave-one-out cross-validation or k-fold cross-validation, can be performed, with the sole proviso that this is also done separately for the two measurement locations A, B. In the illustrated embodiment, a number of "out-of-sample" predictions is obtained for each measurement location A, B, which corresponds to the number of measurement data sets MDS of the respective learning data collection LA, LB. This, again, is not mandatory.

[0125] Furthermore, in step A5, separate location-specific prediction group learning models MA1, MA2, MB1, MB2 are trained for each measurement location without using a resampling method, whereby the training is carried out with all location-specific group sub-measurement data sets GUM and associated location-specific target variables T. Here, too, the training of the location-specific prediction group learning models MA1, MA2, MB1, MB2 is optional.

[0126] In step A4, the location- and group-specific predictions AP1, AP2, BP1, BP2 and associated location-specific target variables T from the two different measurement locations A, B are then merged (“data pooling”), and with this data from both measurement locations A, B, a non-resampling-based merging cross-location and cross-group meta-learning model MA+B is trained. The merging of the location- and group-specific predictions AP1, AP2, BP1, BP2 and associated location-specific target variables T can, for example, be done by measurement location A passing its predictions AP1, AP2 and target variables T to measurement location B, or vice versa. Of course, both measurement locations A, B can also pass the data to a third party, which then uses it together for training.

[0127] Data from different measurement sites are generally heterogeneous. In particular, there are systematic differences between the measurement setups used—in the case of MRI images, for example, scanners—as well as the data collection parameters [Chen 2021, Mali et al.]. The training procedure shown in Figure 3 enables harmonization here, since the predictions of site-specific group learning models are focused on the target variable itself and not on the site-specific associations between the measurement data entries MDE and the target variable T. For example, if the intensity of each brain parcel is higher for measurement site A than for measurement site B, this would lead to different site-dependent representations using standard data pooling methods, such as determining the arithmetic mean.However, using the predictions of site-specific group learning models, these are now always predicted target variable values ​​and thus aligned with the target variable. They no longer include these differences in intensity. On the other hand, a high degree of privacy is ensured despite the cross-site data, since no sharing of the raw data is required; instead, only the predictions AP1, AP2, BP1, BP2 from the different measurement sites A and B are combined. This means that collaborations between different sites A and B are possible without sharing raw data or other protected data.

[0128] The following applies to a subsequent prediction (cf. Figure 3 below), whereby the differences to the embodiment of the prediction method according to the invention according to Figure 1 are again explained and reference is made to the above description.

[0129] In step B2, the obtained measurement site-specific prediction group learning models MA1, MA2, MB1, MB2 (alternatively or additionally, the measurement site-specific prediction group learning models MA1, MA2, MB1, MB2, which is not additionally shown in Figure 3 for reasons of clarity) and the obtained merging cross-measurement site and cross-group meta-learning model MA+B are provided. The group sub-measurement data sets GUM of the measurement data set MDS provided in step B1 are fed to the measurement site-specific prediction group learning models MA1, MA2, MB1, MB2 in step B4. The group sub-measurement data sets GUM are fed to both the prediction group learning models MA1 , MA2 of measurement location A and the prediction group learning models MB1 , MB2 of measurement location B, so that for each measurement location A, B measurement location- and group-specific predictions P1 , P2 are obtained (also hatched accordingly).In the present example, a predicted patient age is obtained for each group, i.e. each brain parcel, once using the models MA1, MA2 trained with the data LA from measurement location A and once using the models MB1, MB2 trained with the data LB from measurement location B.

[0130] The predictions AP1, AP2, BP1, BP2 belonging to a group G1, G2 from the two different measurement locations A, B are combined using a statistical procedure, in this case by calculating their mean, so that exactly one group-specific prediction P1, P2 is obtained for each group across all measurement locations. In other words, the mean of AP1 and BP1 as well as the mean of AP2 and BP2 are calculated.

[0131] The (only) group-specific predictions P1 , P2 obtained by averaging are fed to the non-resampling-based merging cross-site and cross-group meta-learning model MA+B, from which a cross-site and cross-group prediction P - in the present example again a patient age - is obtained.

[0132] Figure 4 shows a further embodiment of a prediction method according to the invention, which uses learning models obtained according to the training method from Figure 3, i.e. using cross-site data. As can be seen, the difference is that no average is calculated from the measurement site- and group-specific predictions AP1, AP2, BP1, BP2, but rather these are fed, site by site, to the consolidating meta-learning model MA+B that encompasses all measurement sites and groups, so that measurement site-specific, cross-group predictions PA, PB are obtained. These are then combined using a statistical method, in particular averaging, so that a cross-measurement site- and cross-group prediction P is obtained from PA and PB.

[0133] Figure 5 shows another embodiment of a computer-implemented training method according to the invention for the cross-site case. The difference from the example shown in Figure 3 is essentially that the data from the two different measurement locations A and B are merged at a later point in time. This can also be referred to as "late data fusion," whereas the example shown in Figure 3, an "intermediate data fusion," already corresponds to the predictions of LevelO.

[0134] The scenario in Figure 5 above corresponds to the one in Figure 3 until the site- and group-specific predictions AP1, AP2, BP1, BP2 are obtained. However, these are not merged but further processed separately, e.g., further at the respective site A, B.

[0135] In step A4, a resampling-based, site-specific, cross-group meta-learning model is then trained and tested for each measurement location, wherein the resampling method includes at least one training with a portion of the site- and group-specific predictions AP1, AP2, BP1, BP2 and the associated site-specific target variables T of the learning data collections LA, LB, and at least one testing with another portion of the site- and group-specific predictions AP1, AP2, BP1, BP2 and the associated site-specific target variables T of the learning data collections LA, LB, and wherein the at least one testing provides site-specific cross-group predictions PA, PB.

[0136] Furthermore, in particular also in step A4, a non-resampling-based, site-specific, cross-group meta-learning model MA, MB is trained for each measurement location A, B, wherein the training is carried out with the measurement location- and group-specific predictions AP1, AP2, BP1, BP2 and the associated measurement location-specific target variables T of the respective learning data collection LA, LB, preferably with all measurement location- and group-specific predictions AP1, AP2, BP1, BP2 and associated measurement location-specific target variables T.

[0137] The trained, non-resampling-based, site-specific, cross-group meta-learning models MA, MB are fed with site- and group-specific predictions from the other site A, B, along with the associated target variable T, thereby obtaining site-specific, cross-group cross-predictions PAB, PBA. In other words, model MA is fed with the predictions BP1 and BP2 and receives the predictions PAB, and model MB is fed with the predictions AP1 and AP2 and receives the predictions PBA. Accordingly, the predictions PAB are those obtained using a model belonging to site A and data from site B, and vice versa for PBA.

[0138] Using the site-specific cross-group predictions PA, PB and the site-specific cross-group predictions PAB, PBA and the corresponding target variables T of the learning data sets LA, LB, a non-resampling-based cross-site meta-learning model MA_B is then trained so that it can provide a site- and cross-group prediction P from several site-specific cross-group predictions PA, PB.

[0139] A corresponding embodiment of a computer-implemented prediction method according to the invention is shown in Figure 5 below. This corresponds to those according to Figures 3 and 4 until the measurement site- and group-specific predictions AP1, AP2, BP1, BP2 are obtained. Then, however, the non-resampling-based measurement site-specific cross-group meta-learning models MA, MB are used. Specifically, the group-specific predictions of the corresponding measurement site A, B are fed to these, i.e., the predictions AP1 and AP2 are fed to the trained model MA, and the learned predictions BP1 and BP2 are fed to the model MB. The cross-group measurement site-specific prediction PA is obtained from the model MA, and the cross-group measurement site-specific prediction PB is obtained from the model MB (one patient age from each measurement site-specific model).These predictions PA, PB are then fed to the non-resampling-based cross-site meta-learning model MA_B, which provides the prediction P, in this case a patient age.

[0140] Figure 6 shows another embodiment of a cross-site scenario. This scenario uses an additional learning data collection LC from a measurement location C that is different from measurement locations A and B. It should be noted that in addition to the steps shown in Figure 6, steps of the training procedure shown in Figure 5 must be completed, namely at least those for obtaining the models MA1, MA2, MB1, MB2, as well as MA and MB, each using the corresponding learning data collections LA and LB, respectively.

[0141] As can be seen in Figure 6, the measurement data entries MDE of the measurement data sets MDS of the additional learning data collection LC are then grouped into several groups G1, G2 as provided in step A2, whereby additional group sub-measurement data sets GUM are obtained.

[0142] The additional group sub-measurement data sets GUM and the target variables T of the additional learning data collection LC are then fed to the trained non-resampling-based site-specific group learning models MA1, MA2, MB1, MB2 belonging to the measurement locations A and B, respectively, and from these site- and group-specific predictions are obtained, namely AP1, AP2 on the one hand and BP1, BP2 on the other hand.

[0143] The site- and group-specific predictions AP1, AP2, BP1, BP2 are fed to the trained, non-resampling-based site-specific cross-group meta-learning models MA, MB belonging to the sites A, B, respectively, and from these, site-specific cross-group predictions PA, PB are obtained. This is somewhat analogous to Figure 5, except that AP1, AP2, BP1, BP2 are obtained with the models MA1, MA2, MB1, MB2, but not using data from these sites A, B, but rather from the independent site C.

[0144] A non-resampling-based additional cross-site meta-learning model MA_Bc is then trained using the site-specific cross-site predictions PA, PB and the corresponding target variables T of the additional learning data set LC from site C.

[0145] As can be seen from Figure 6 below, a corresponding embodiment of a related computer-implemented prediction method corresponds to the example in Figure 5, with the sole proviso that the non-resampling-based additional cross-site meta-learning model MA_Bc is used to obtain P. Such a system offers many advantages for federation learning-based collaborations between multiple working groups. An example is a scenario in which site / hospital A and site / hospital B internally apply the same training procedure to then share predictions on site C with each other or a third party, e.g., researchers. This means that neither the trained models nor the raw data need to be shared. This allows maximum protection for the security and privacy of the data from sites A and B. Site C can be a dataset accessible to all parties or even publicly.This means that both private and public or other shared datasets, large and small, can be ideally utilized. An example of such a now more usable large-scale dataset is the UK Biobank (UKB) [Sudlow et al. 2015]. Of course, this exact scenario is only an example, and the use of a different number of both private and shared datasets from different measurement sites is possible.

[0146] The steps of the above-described embodiments of both the training and prediction methods according to the invention can each be carried out using at least one computer. One or more computer programs with program code means can be used which, when executed on at least one computer, cause the at least one computer to carry out the above steps. The use of a (private and / or public) cloud is also possible.

[0147] REFERENCES

[0148] Chen et al. 2021 : Chen, J., Müller, V.I., Dukart, J., Hoffstaedter, F., Baker, J.T., Holmes, A.J., Vatansever, D., Nickl-Jockschat, T., Liu, X., Derntl, B., et al. (2021). Intrinsic Connectivity Patterns of Task-Defined Brain Networks Allow Individual Prediction of Cognitive Symptom Dimension of Schizophrenia and Are Linked to Molecular Architecture. Biol. Psychiatry 89, 308-319.

[0149] Finn et al. 2015: Finn, E.S., Shen, X., Scheinost, D., Rosenberg, M.D., Huang, J., Chun, M.M., Papademetris, X., and Constable, R.T. (2015). Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity. Nat. Neurosci. 18, 1664-1671

[0150] Hedderich & Eickhoff 2021 : Hedderich, D.M., and Eickhoff, S.B. (2021).

[0151] Machine learning for psychiatry: getting doctors at the black box? Mol. Psychiatry 26, 23-25

[0152] Kaissis et al. 2020: Kaissis, G.A., Makowski, M.R., Rückert, D., and

[0153] Braren, R.F. (2020). Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell

[0154] Larabi et al. 2021 : Larabi, D.I., Gell, M., Amico, E., Eickhoff, S.B., and Patil, K.R. (2021). Highly accurate local functional fingerprints and their stability. BioRxiv

[0155] http: / / dx.doi.org / 10.1037 / 0033-295X.103.2.2021 Mali et al. Making Radiomics More Reproducible Across Scanner and Imaging Protocol Variations: A Review of Harmonization Methods. J. Pers. Med. 11 Myszczynska et al. 2020: Myszczynska, MA, Ojamies, PN, Lacoste, AMB, Neil, D., Saffari, A., Mead, R., Hautberge, GM, Holbrook, JD, and Ferraiuolo, L. (2020). Applications of machine learning to the diagnosis and treatment of neurodegenerative diseases. Nat. Rev. Fr. Neurol. 16, 440–456 Schaefer et al. 2018: Schaefer , A. , Kong , R. , Gordon , EM , Laumann , 2018 .

[0156] TO , Zuo , X.-N. , Holmes , AJ , Eickhoff , SB , and Yeo , BTT (2018). Local- Global Parcellation of the Human Cerebral Cortex from Intrinsic Functional Connectivity MRI. Cereb. Cortex 28, 3095–3114

[0157] Sudlow C et al. 2015 : Sudlow , C. , Gallacher , J. , Allen , N. , Beral , V. , Burton , P. , Danesh , J. , Downey , P. , Elliott , P. , Green , J. , Landray , M. , et al. (2015). The UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 12 , e1

[0158] Yeo et al. 2011 : Yeo , BTT , Krienen , FM , Tomb , J , Sabuncu , MR , Lashkari , D , Hollinshead , M , Roffman , JL , Smoller , JW , Zöllei , L , Poli- meni , JR , et al. (2011). The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J. Neurophysiol. 106 , 1125–1165

Claims

REQUIREMENTS 1. Computer-implemented training method for training machine learning models, in which A1) at least one learning data collection (L, LA, LB, LC) with multiple learning data sets (LDS) is provided, wherein each learning data set (LDS) comprises a measurement data set (MDS) with multiple measurement data entries (MDE) and a target variable (T) assigned to the measurement data set (MDS), preferably, wherein all learning data sets (LDS) have the same structure, characterized in that A2) a grouping of at least some of the measurement data entries (MDE) of the measurement data sets (MDS) of the at least one training data collection (L, LA, LB, LC) into several groups (G1, G2) is carried out, thereby obtaining group sub-measurement data sets, wherein the group sub-measurement data sets (GUM) of a group (G1, G2) each comprise corresponding measurement data entries (MDE) from different measurement data sets (MDS), A3) for each group (G1, G2) a separate machine learning model is trained and tested using a resampling method, resampling-based group learning model (RsB MG1, RsB MG2), wherein the resampling-based method includes at least one training with a subset of the group sub-measurement datasets (GUM) belonging to the respective group (G1, G2) and the associated target variables (T), and at least one testing with another subset of the group sub-measurement datasets (GUM) belonging to the respective group (G1, G2), and wherein the we- at least one test provides predictions for different groups, group-specific predictions (P1 , P2, AP1 , AP2, BP1 , BP2), A4) with the group-specific predictions (P1 , P2, AP1 , AP2, BP1 , BP2) and target variables (T) of at least one training data set (L, LA, LB, LC) at least one further machine learning model, cross-group meta-learning model (MM, MA+B, MA_B), is trained, in particular without using a resampling method, so that it can provide a cross-group prediction (P) from several group-specific predictions (P1 , P2, AP1 , AP2, BP1 , BP2) assigned to several different groups (G1 , G2), A5) optionally, for each group (G1, G2), a further machine learning model is trained with group sub-measurement datasets (GUM) and associated target variables (T) belonging to the respective group (G1, G2), in particular without using a resampling method, prediction group learning model (MG1, MG2), wherein the training is preferably carried out with all group sub-measurement datasets (GUM) and associated target variables (T) belonging to the respective group (G1, G2).

2. Computer-implemented training method according to claim 1, characterized in that the different learning data sets (LDS) belong to different patients and the learning data sets (LDS) comprise medically relevant measurement data sets (MDS), in particular measurement data sets (MDS) obtained by means of medical diagnostic procedures, and the target variables (T) relate to or are given by properties of the patients, in particular an age and / or a disease state of the patients.

3. Computer-implemented training method according to claim 2, characterized in that the measurement data sets (MDS) are generated by image acquisition. images, in particular MRI images, of human brains or parts thereof, and the measurement data entries (MDE) each correspond to or are assigned to a voxel, and that a grouping into brain parcels takes place.

4. Computer-implemented training method according to claim 1, characterized in that the different learning data sets (LDS) belong to different persons and the measurement data sets (MDS) are given by or comprise image recordings of at least a part of the face and / or at least a part of the body of the persons, and the target variables (T) relate to or are given by properties of the persons, preferably wherein the measurement data entries (MDE) each correspond to or are assigned to a pixel, and / or wherein the properties of the persons do not relate to or are not given by a disease state of the persons.

5. Computer-implemented training method according to claim 1, characterized in that the various learning data sets (LDS) belong to different sections of the Earth's surface and the measurement data sets (MDS) are provided by or comprise image recordings, in particular satellite images, of the Earth's surface sections, and the target variables (T) relate to or are provided by properties of the Earth's surface sections, in particular the presence of certain elements, preferably the presence of fields and / or rivers and / or lakes.

6. Computer-implemented training method according to claim 1, characterized in that the different learning data sets (LDS) belong to different persons and the measurement data sets (MDS) are supplemented by information about the user behavior of the persons, in particular on at least one website, are given or include such, and the target variables (T) relate to or are given information about actions taken by the persons, in particular purchases made by the persons.

7. Computer-implemented training method according to claim 1, characterized in that the various learning data sets (LDS) belong to different DNA sequences and / or protein sequences and / or gene expressions and the measurement data sets (MDS) are given by or comprise information about the DNA sequences and / or protein sequences and / or gene expressions, in particular by the structure of such information, and the target variables (T) relate to or are given by features of the DNA sequences and / or protein sequences and / or gene expression data, in particular binding sites and / or protein-protein interactions and / or solvent properties thereof.

8. Computer-implemented training method according to one of the preceding claims, characterized in that in step A2 a user-defined grouping or a grouping of the measurement data sets (MDS) derived from the at least one learning data collection (L, LA, LB, LC), in particular the structure of the measurement data sets (MDS), preferably obtained by clustering, is carried out.

9. Computer-implemented training method according to one of the preceding claims, characterized in that in step A1 learning data collections (L, LA, LB LC) are provided from and in particular at different measurement locations (A, B, C), wherein the learning data collections (L, LA, LB LC) each comprise several learning data sets (LDS) with measurement data entries (MDE) and associated target variables (T).

10. Computer-implemented training method according to claim 9, characterized in that in step A3, for each measurement location (A, B, C), separate resampling-based measurement-location-specific group learning models are trained and tested with the learning data sets (LDS) belonging to the respective measurement location (A, B, C), wherein the resampling method includes at least one training with a part of the measurement-location-specific group sub-measurement data sets (GUM) and associated measurement-location-specific target variables (T), and at least one testing with another part of the measurement-location-specific group sub-measurement data sets (GUM), and wherein the at least one testing for the respective measurement location (A, B, C) provides predictions for different groups, measurement-location- and group-specific predictions (AP1 , AP2, BP1 , BP2).

11. Computer-implemented training method according to claim 9 or 10, characterized in that in step A5, for each measurement location (A, B, C), separate measurement-location-specific prediction group learning models (MA1 , MA2, MB1 , MB2) are trained, in particular without using a resampling method, wherein the training preferably takes place with all measurement-location-specific group sub-measurement data sets (GUM) and associated measurement-location-specific target variables (T).

12. Computer-implemented training method according to one of claims 9 to 11, characterized in that in step A4 the measurement site and group-specific predictions (AP1 , AP2, BP1 , BP2) and associated measurement site-specific target variables (T) of different measurement sites (A, B, C) are combined and thus a combined measurement site and group-spanning meta-learning model (MA+B) is trained, in particular without the use of a resampling method.

13. Computer-implemented training method according to one of claims 9 to 12, characterized in that in step A4 a resampling-based, location-specific, cross-group meta-learning model is trained and tested for each measurement site (A, B, C), wherein the resampling method includes at least one training with a part of the location- and group-specific predictions (AP1, AP2, BP1, BP2) and the associated location-specific target variables (T) of the training data collections (L, LA, LB, LC), and at least one testing with another part of the location- and group-specific predictions (AP1, AP2, BP1, BP2) and the associated location-specific target variables (T) of the training data collections (L, LA, LB, LC), and wherein the at least one testing provides location-specific cross-group predictions (PA, PB).

14. Computer-implemented training method according to claim 13, characterized in that a cross-measurement-location meta-learning model (MA_B) is trained with the measurement-location-specific cross-group predictions (PA, PB) originating from several measurement locations (A, B, C) and associated measurement-location-specific target variables (T), in particular without using a resampling method, so that it can provide a measurement-location and cross-group prediction (P) from several measurement-location-specific cross-group predictions (PA, PB).

15. Computer-implemented training method according to one of claims 9 to 14, characterized in that, in particular in step A4, a measurement-site-specific, cross-group meta-learning model (MA, MB) is trained for each measurement location, in particular without using a resampling method, wherein the training is carried out with the measurement-site- and group-specific predictions (AP1, AP2, BP1, BP2) and the associated measurement-site-specific target variables (T) of the respective training data. Data collection (L, LA, LB, LC) is performed, preferably with all measurement site and group-specific predictions (AP1 , AP2, BP1 , BP2) and associated measurement site-specific target variables (T).

16. Computer-implemented training method according to claims 11 and 15, characterized in that an additional training data set (LC) is provided from an additional measurement location (C) different from the measurement locations (A, B), and the measurement data entries (MDE) of the measurement data sets (MDS) of the additional training data set (LC) are grouped into several groups (G1, G2) as provided in step A2, thereby obtaining additional group sub-measurement data sets (GUM), the additional group sub-measurement data sets (GUM) and the target variables (T) of the additional training data set (LC) are each fed to the measurement location-specific prediction group learning models (MA1, MA2, MB1, MB2) belonging to the measurement locations (A, B), in particular without the use of a resampling method, and measurement location- and group-specific predictions (AP1, AP2, BP1, BP2) are obtained from these, which Location- and group-specific predictions (AP1 , AP2, BP1 ,BP2) are fed to the measurement site-specific, cross-group meta-learning models (MA, MB) belonging to the measurement sites (A, B), which are trained in particular without the use of a resampling method, and measurement site-specific, cross-group predictions (PA, PB) are obtained from these, and an additional cross-measurement site meta-learning model MA_Bc is trained with the measurement site-specific, cross-group predictions (PA, PB) and the target variables (T) of the additional learning data collection (LC), in particular without the use of a resampling method.

17. Computer-implemented training method according to claim 15 or 16, characterized in that the trained measurement site-specific cross-group meta-learning model (MA, MB) of at least one measurement site (A, B, C) is supplied with measurement site- and group-specific predictions (AP1 , AP2, BP1 , BP2) of at least one other measurement site (A, B, C) and thereby measurement site-specific cross-group predictions (PAB, PBA) are obtained.

18. Computer-implemented training method according to claim 14 and 17, characterized in that the cross-measurement site meta-learning model MA_Bc is trained with the measurement site-specific cross-group predictions (PA, PB) and the measurement site-specific cross-group predictions (PAB, PBA).

19. Computer-implemented prediction method for predicting a property using machine learning models obtained by performing the training method according to one of the preceding claims, wherein B1) a measurement data set (MDS) with multiple measurement data entries (MDE) is provided, B2) predictive group learning models (MG1 , MG2, MA1 , MA2, MB1 , MB2) trained according to step A5 and / or resampling-based group learning models (RsB MG1 , RsB MG2) trained according to step A2 and at least one cross-group meta-learning model (MM, MA+B, MA_B) trained according to step A4 in particular without using a resampling method are provided, B3) the measurement data entries (MDE) of the measurement data set (MDS) provided in step B1 are grouped analogously to step A2, which results in a group sub-measurement data set (GUM) for each group (G1, G2), B4) the group sub-measurement datasets (GUM) are each fed to the corresponding prediction group learning models (MG1 , MG2, MA1 , MA2, MB1 , MB2) or the corresponding resampling-based group learning models (RsB MG1 , RsB MG2) and a group-specific prediction (P1 , P2) is obtained from each of these as output, B5) the group-specific predictions (P1 , P2, AP1 , AP2, BP1 , BP2) are fed to at least one cross-group meta-learning model (MM, MA+B, MA_B) trained in particular without the use of a resampling method and a cross-group prediction (P) is obtained from this as output.

20. Computer-implemented prediction method according to claim 19, wherein - in step B2, location-specific prediction group learning models (MA1, MA2, MB1, MB2) of different locations (A, B, C) obtained by performing the training method according to claim 11, and / or resampling-based location-specific group learning models of different locations (A, B, C) obtained by performing the training method according to claim 10, and a merging location- and group-spanning meta-learning model (MA+B) obtained by performing the training method according to claim 12, in particular without using a resampling method, are provided. - the group sub-measurement data sets (GUM) obtained in step B3 in Step B4 the measurement location-specific prediction group Learning models (MA1, MA2, MB1, MB2) or the resampling-based measurement site-specific group learning models of the various Measurement locations (A, B, C) are supplied and measurement location- and group-specific predictions (AP1 , AP2, BP1 , BP2) are obtained, - the predictions (AP1, AP2, BP1, BP2) belonging to each group (G1, G2) from different measurement locations (A, B, C) are combined using a statistical procedure, in particular averaging, so that a group-specific prediction (P1, P2) is obtained for each group (G1, G2), - the group-specific predictions (P1 , P2) are fed into the merging measurement-site and group-spanning meta-learning model (MA+B) and a measurement-site and group-spanning prediction (P) is obtained from this.

21. Computer-implemented prediction method according to claim 19 or 20, wherein in step B2 location-specific prediction group learning models (MA1, MA2, MB1, MB2) of different locations (A, B, C), obtained by performing the training method according to claim 11, and / or resampling-based location-specific group learning models of different locations (A, B, C), obtained by performing the training method according to claim 10, and a merging location- and group-spanning meta-learning model (MA+B), obtained by performing the training method according to claim 12, in particular without using a resampling method, are provided, and the group sub-measurement data sets (GUM) obtained in step B3 are provided in step B4 to the location-specific prediction group learning models (MA1, MA2, MB1, MB2) or the resampling-based location-specific group learning models of the different measurement locations (A, B, C).C) are supplied and measurement location and group-specific predictions (AP1 , AP2, BP1 , BP2) are obtained, - the measurement site- and group-specific predictions (AP1 , AP2, BP1 , BP2) are fed measurement site by measurement site into the merging measurement site- and group-spanning meta-learning model (MA+B) and thus measurement site-specific group-spanning predictions (PA, PB) are obtained for each measurement site (A, B, C), - the measurement site-specific cross-group predictions (PA, PB) are combined using a statistical method, in particular averaging, so that a measurement site and cross-group prediction (P) is obtained.

22. Computer-implemented prediction method according to one of claims 19 to 21, wherein - in step B2, location-specific prediction group learning models (MA1, MA2, MB1, MB2) of different locations (A, B, C) obtained by performing the training method according to claim 11, and / or resampling-based location-specific group learning models of different locations (A, B, C) obtained by performing the training method according to claim 10, and location-specific cross-group meta-learning models (MA, MB) obtained by performing the training method according to claim 15, in particular without using a resampling method, and a cross-location meta-learning model (MA_B) obtained by performing the training method according to claim 14, in particular without using a resampling method, and / or an additional cross-location meta-learning model (MA_Bc) obtained by performing the training method according to claim 16, in particular without using a resampling method, are provided. - the group sub-measurement data sets (GUM) obtained in step B3 are used in step B4 to create the measurement location-specific prediction group- Learning models (MA1, MA2, MB1, MB2) of the different measurement locations (A, B, C) are fed and measurement location- and group-specific predictions (AP1, AP2, BP1, BP2) are obtained. - the measurement site- and group-specific predictions (AP1 , AP2, BP1 , BP2) are fed to the respective associated measurement site-specific group-spanning meta-learning model (MA, MB) on a measurement site-by-measurement basis, thus obtaining measurement site-specific group-spanning predictions (PA, PB) for each measurement site (A, B, C), - the measurement site-specific cross-group predictions (PA, PB) are fed to the cross-measurement site meta-learning model (MA_B) or the additional cross-measurement site meta-learning model (MA_Bc) and a measurement site and group-spanning prediction (P) is obtained from it.

23. Computer program comprising program code means which, when executed on at least one computer, cause the at least one computer to perform the steps of the training method according to any one of claims 1 to 18 and / or the steps of the prediction method according to any one of claims 19 to 22.

24. Computer-readable medium comprising instructions which, when executed on at least one computer, cause the at least one computer to perform the steps of the training method according to any one of claims 1 to 18 and / or the steps of the prediction method according to any one of claims 19 to 22.

25. Device for carrying out the computer-implemented training method according to one of claims 1 to 18 and / or the computer-implemented prediction method according to one of claims 19 to 22, comprising a processor and a data storage device on which the computer program according to claim 23 is stored.

Images