System for training ensemble neural network devices to assess forecast uncertainty

Abstract

A system (200) for training an ensemble neural network device, comprising the steps of: - providing (205) a sample data set comprising at least one input set (220) and at least one output set (225) associated with the input set to a neural network device comprising an ensemble (230) of neural network devices configured to provide independent predictions based on the sample data; - operating (210) a neural network device on a sample data set; - obtaining a trained neural network device (215) configured to provide an output; A system (200) configured to execute the neural network device further comprises at least two independent activation functions, at least two of the independent activation functions representing a statistical distribution of a plurality of independent predictions, the neural network device being configured to provide at least one output (235, 236) for the at least two of the independent activation functions; - the operating step further comprises operating each neural network device of the ensemble to provide an ensemble of outputs, the neural network devices being trained to minimize values ​​representing at least two of said independent activation functions; A system (200).

Description

[Technical field]

[0001] The present invention relates to a system for training a neural network device, a computer-implemented method for training a neural network device, a corresponding computer-implemented neural network device, a computer program product for performing the steps of the method that is the subject of the present invention, a computer readable medium storing instructions for performing the steps of the method that is the subject of the present invention, a computer-implemented method for predicting physical, chemical, medical, sensory or pharmaceutical properties of flavour, fragrance and drug ingredients, and a computer-implemented method for predicting representational categories in images.

[0002] The invention finds particular application in the flavour, fragrance and drug industries, applicable to the biological, chemical, medical and pharmaceutical industries, or in the imaging industry. [Background technology]

[0003] In scientific experiments, measurements 105 stored in a database such as that shown in FIG. 1 vary depending on the environment of the experiment. Typically, an ideal environment such as the International Space Station needs to be used to obtain stable conditions. Even under near perfect environmental conditions, technicians may prepare instruments and samples slightly differently from each other. Merging experimental data from different sources can be difficult because the variance from experiment to experiment varies depending on the measurement conditions used. To reduce such variance and achieve moderate to good success, statistical methods have been developed to homogenize experimental data. In the field of machine learning, such variance can occur between training and test sets, and between known and future data.

[0004] Another widely known problem with machine learning models is the number of hyperparameters in the model, which can significantly affect the model's ability to overfit to the training data 110 as shown in FIG. 1. The amount of available data may be insufficient to train a large number of hyperparameters in a neural network. Therefore, it is best to consider the minimum amount of data necessary to extract a meaningful digital data representation for the data. This can be achieved by privileged layers with a smaller number of parameters. One example of such an attempt is to replace dense layers with compact convolutional layers. Convolutional layers are particularly specialized to extract localized features from data, such as images. However, even with compact and efficient convolutional layers, networks with billions of parameters can still be generated. Trend analysis of megamodels shows that such models can only be trained with extremely large datasets available, for example, for images, literature, and music. In contrast, scientific fields such as chemistry, biology, etc. often have datasets containing hundreds to thousands of data points. Therefore, in these domains, the goal should be to reduce the number of parameters to fit the data size.

[0005] One approach to compensate for large network sizes is through data augmentation. Indeed, the increase in performance with increasing augmentation rate indicates that more data is needed for the size of the network chosen, suggesting that the size of the network should be reduced as much as possible. At the same time, the augmentation rate can be used to identify whether the network is critically parameterized, i.e., the point at which augmentation has no or little effect on the model's performance. Not all models are open to data augmentation. For example, graph neural networks (GNNs) are invariant to shuffling of representations. Thus, GNNs are not compatible with existing data augmentation methods used for natural language processing or images.

[0006] The third problem is that the model training procedure 115 determines a key aspect in modeling. Frequently, there are too many variables that can affect the model decision. This may explain why hyperparameterization optimization strategies may be required to improve the model for performance or efficiency. Apart from the selected model, issues regarding the data split between the training and test sets also play an important role. Multiple methods of cross-validation, from full leave-one-out random split to K-fold, can be used to simulate and estimate the model quality on unseen data. In the end, the model prediction is just a supervised estimate that depends on the training conditions, model size, optimization parameters and data split used. At the completion, it is not possible to know if the best model was actually trained. However, here it is assumed that the model calculated is the best model for the test points used. This is a general limitation of the data modeling approach, since one cannot necessarily expect the performance results to be the same for all parts of future unseen data. It should also be noted that future performance may vary considerably depending on the sample size evaluated against unseen data and possible sample biases introduced into the unseen data. One approach to partially address these shortcomings is by predicting an accurate theoretical endpoint as a standardized metric for evaluating the model. An example of such an endpoint is the molecular weight of a molecule in a chemical reaction.

[0007] Over the past few decades, deep neural networks (DNNs) have become an essential part of modern life. Besides being a key technology in many scientific fields, DNNs are now increasingly being offered as solutions for real-world applications. Neural networks are already being continuously applied in the fields of medical imaging, image recognition and autonomous vehicles. Due to their rapid growth in many applications, it is important to consider some of the significant drawbacks of neural networks. These drawbacks mainly include: -Lack of expressiveness and transparency in the model's decision logic, - the inability to distinguish between in-domain and out-of-domain predictions, including sensitivity to domain shifts; - sensitivity to adversarial examples, which results in neural networks that may be vulnerable to adversarial attacks; and / or - the inability to provide reliable model uncertainty estimates; Includes:

[0008] Depending on the scientific domain, the demand to provide explanations for the predictions of neural networks is a topic that is frequently discussed. Indeed, deep neural networks are typically perceived as black-box technologies. However, this should not be considered an essential drawback for their acceptance in applications. Furthermore, it should be noted that multiple comparable performing solutions can be generated, and the estimated logic of the machine can be varied. This applies in particular between identical "machines" or between models with different random initializations. Thus, the explanations can vary between "machines", and as a result, it is not possible to tell if the explanation is the best possible explanation. In other words, the estimated logic is a property of the trained system and is one possible explanation.

[0009] It is widely known in machine learning and artificial intelligence that the predictive performance of a model can vary significantly between in-domain and out-of-domain points. In the case of a linear regression on a set of points (x,y), the modeled mathematical function y=f(x) should only be applied to all points that fall between the known points used to define this function. Out-of-domain predictions are extrapolations to known data, and the behavior of the outside data is unknown. It is proposed here (e.g. in the publication Hanser & al., Applicability Domain: Towards a More Formal Framework to Express the Applicability of a Model and the Confidence in Individual Predictions, Advances in Computational Toxicology, pp 215-232) that such predictions should be systematically flagged or discarded. The domain of applicability of a model is furthermore simultaneously defined by the architecture of the model and the data used to train it. Thus, the system with the combined architecture and data defines a closed system and is an example of an indeterminate problem. Therefore, this one system cannot be used to simultaneously determine the contributors of the out-of-domain prediction, i.e., whether it comes from the architecture or the data. Thus, this does not mean that the prediction will be out-of-domain for all systems. To obtain the out-of-domain evaluation, mechanisms such as distance to the training set, statistical hotelling on the latent space, or the use of evidence layers are typically applied. Note that for these metrics, the in-domain and out-of-domain calls are the results from the model and the selected domain method. In a stricter sense, it can also be considered that the fact that the prediction is out-of-domain by one selected out-of-domain evaluation method does not mean that the point is out-of-domain for all evaluation methods.In summary, users with limited confidence in the model's predictions may raise similar concerns regarding the methods used for out-of-domain evaluation or the methods for explanations provided. In a regressional approach, this process can continue indefinitely. Unlike providing explanations, the use of out-of-domain evaluation can be considered as an important value generated for prediction. Such metrics are expected to provide sufficient information to guide the decision-making process, which is particularly important for sciences with high experimental costs or strict regulatory acceptance criteria, such as the chemical, pharmaceutical and biological industries.

[0010] Neural networks are frequently reported to be vulnerable to adversarial attacks, i.e., they can be subject to sabotage such as deliberate call reversals or the addition of noise to the input representation. In addition to safety concerns, adversarial examples also have a prescribed effect in applications that involve significantly lower levels of risk. An adversarial example is usually defined as a high-confidence misclassification by the network. The most promising solutions are those that use Monte Carlo dropout networks. However, it should be noted that even if one class can be said to exist for any image under consideration, the set of classes reported may be incomplete, since images may be perceived differently between humans. Thus, it is not always possible to distinguish whether a negative representation originates from a true negative call ("absolute negation") or from a missed call ("missed mention"). In the first solution, Bayesian neural networks (BNNs) have been used to evaluate the importance of uncertainty to adversarial examples. The evaluated BNNs display significant shortcomings with respect to the concept of representation learning. In fact, BNNs with Bayesian logic are typically reported to be less traceable than other neural networks. Therefore, no judgement can be made on the quality of the learned representation, and thus, itself, an example of the indefinability theorem mentioned above. Thus, a meta-analysis is required to form a judgement call on the quality of the learned representation. It has been reported that Monte Carlo dropout networks, while being significantly more robust against adversarial examples than their equivalent deterministic counterparts, typically underestimate the uncertainty level. In conclusion, despite the recognition that uncertainty and robustness are key points for combating adversarial examples, the quantification of the uncertainty level remains unsolved.

[0011] Recently, and especially notably, adversarial neural networks are often used for generative purposes in the context of deepfakes. The most widely known example of such a network is the generative adversarial network (GAN). A typical GAN ​​consists of a system of two single deterministic networks: a generative neural network G and a discriminator D. Similar to the adversarial competition between law enforcement and criminal intent, the GAN is trained applying an alternating mechanism that optimizes the generator G and the discriminator D. The alternating mechanism is applied here with the goal of enabling the generator G to generate realistic examples that can no longer be distinguished from real-world data by the discriminator D. GANs have been trained adaptively using an unsupervised discriminator. Regardless of the training mechanism chosen, G and D define a single "machine" system. As a result, the quality and truth of the converged solution is unprovable within the system itself. In other words, the fact of convergence of the model is not enough to prove that the generated output is indeed true to reality. Therefore, a meta-analysis is needed to evaluate whether the quality of such neural networks is within expected limits, which can be used in production as well. An example of such a meta-analysis would be to ask a representative set of users to select real pictures from a mixed set of real and generated images.

[0012] According to the above considerations, indefinability and incompleteness define the main drawbacks of any machine learning model. In other words, the truth of one machine cannot be evaluated within the system itself. If a machine fails to answer a given question, it does not mean that all machines will fail to answer the same set of questions with incorrect answers. Also, if several machines provide correct answers for some questions, it cannot be immediately determined which machine provides better answers for some or all of the questions asked. Ultimately, it is not possible to know if the selected machine is the best solution for all future unseen data. However, we hope that the selected machine is the best solution for many questions. However, we should usually assume that optimal results are obtained for the best performing model. To address these issues deeply, it is essential to evaluate the prediction uncertainty or model uncertainty. Furthermore, a trained model, typically consisting of an architecture, data, and especially some other variables, can be considered to define a closed system. The performance of such a closed system is typically calibrated using the method of cross-validation. Even if the test data is not used for training, it should still be known that the test data is still a sample from the data generated for a specific purpose and known at the moment. This may introduce bias in the test data. Thus, even if the test set is largely representative of a representative set of future unseen data points, one must expect that this is not necessarily true for all future data. That is, results on future data may deviate significantly from those measured on the test set. Especially in selection problems where past choices are used to derive future choices, strong deviations may occur. Results are also strongly affected by sample size and sample bias. Moreover, model validation is already performed during training of the model using the test data and should not be judged using future data.

[0013] Uncertainty in modeling is typically distinguished between aleatoric and epistemic uncertainty. Aleatoric uncertainty is generally understood as statistical uncertainty, i.e., each time an experiment is performed, it is subject to variability caused by unknown mechanisms. More generally, in the present context, it is discussed in terms of standard deviation or data variance. Aleatoric uncertainty is therefore defined by the data used to train the model and is referred to as data uncertainty. It can therefore be expected that as the noise level in the data increases, the prediction error also increases. It is also predictable that data generated using the same procedural representation will be homoscedastic, whereas data coming from multiple sources can be heteroscedastic. Varying variance can also be introduced by experimental conditions that change over time. When data are heteroscedastic, variance is an essential property for successful merging of data from different sources. In contrast to aleatoric uncertainty, epistemic uncertainty refers to the concept of errors caused by technical limitations or lack of technical knowledge in the case of machine learning and deep learning models. Thus, in the context of modeling, epistemic uncertainty can be referred to as model uncertainty. This kind of uncertainty can be easily illustrated by explaining that it typically arises in cases where model parameters are not fully determined, i.e. training is done in the absence of data, also referred to as having extensive a posteriori uncertainty about the network's hyperparameters.

[0014] In recent years, various methods have been proposed to evaluate the uncertainty in deep learning models. It can be noted that the evaluation of uncertainty has been ongoing since the 1990s. However, these early publications talk about data uncertainty, not model uncertainty. In other words, these models aim to predict the variance observed in the data, not the variance between models. A notable drawback of such predictions is the fact that the data variance is typically not a property of the data points. In reality, the uncertainty of the observed data can vary significantly based on the number of points reported, and typically stabilizes to the true variance with an increasing number of measurements. Various methods have been introduced to evaluate the model uncertainty. These methods are typically distinguished into four trends: 1) test time augmentation methods, 2) Bayesian methods, 3) uncertainty of single deterministic neural networks, and 4) ensemble neural networks.

[0015] The evaluation of the uncertainty generated by the test time augmentation method is based on predicting different input representations using the same model. By augmentation, we mean generating different representations of the same object that can be sent to the same neural network. In the case of images, augmentation can be easily exemplified by applying a horizontal or vertical flip or rotation to the image. The network is expected to recognize the object regardless of the representation used. In the present context, we rather speak of the uncertainty of the representation. Therefore, test time augmentation is an essential method for evaluating adversarial examples, i.e., we evaluate whether the modifications have no or limited impact on the model's predictions. Other reports have shown that augmentation should be applied during training and testing time to improve the robustness and stability of neural networks. The main drawback of this method is the requirement that the data representation be augmentable. This is possible for image and language neural networks, but not for any network that generates a latent space that is invariant with respect to shuffling of representations. An example of the latter type of neural network is the graph neural network, which is often used for molecular prediction, as shown, for example, in the paper David Duvenaud & al., Convolutional Networks on Graphs for Learning Molecular Fingerprints (https: / / arxiv.org / abs / 1509.09292).

[0016] In Bayesian neural networks, probability distributions over the parameters of the model are learned, often followed by applying a regularization to the probabilities. Thus, two successive steps create a layer of evidence for essential features. A major drawback of the Bayesian approach includes the fact that the solution does not define a closed-form solution, especially for systems with high levels of complexity. An example of a complex system is a neural network. To address this issue, computationally expensive Approximate Bayesian Inference (ABI) techniques must be applied to the calculation of the posterior probabilities. Alternatively, ABIs can be incorporated into the network. However, this systematically creates exceptionally large neural networks, limiting the practical usefulness of these types of networks. Moreover, Bayesian neural networks also exhibit a low level of traceability, which can be partially corrected by applying approximation techniques such as Monte Carlo approximation. Finally, it is widely known that the success of the Bayesian approach depends on the choice of the relevant prior distribution. This point has not yet been resolved for neural networks. Specifying the optimal prior distribution remains an open problem in the context of deep learning.

[0017] The application of a Monte Carlo dropout strategy is currently the best method available to evaluate model uncertainty, as shown in the paper Lewis Smith & al., Understanding Measures of Uncertainty for Adversarial Example Detection (https: / / arxiv.org / abs / 1803.08533). In this solution, dropout layers are used during the training and inference phases of the model. To calculate the uncertainty, multiple forward passes through the network are required, which defines technical constraints on the method used. Regarding the generated uncertainty values, it has been reported that these values ​​are often too optimistic, and more specifically, they tend to underestimate the uncertainty. This effect can be explained by the fact that for each pass, a subset of the latent variables is randomly selected, and the random selection can repeatedly use the same set of variables. This problem can be improved by using an exceptionally large dropout rate, but this of course leads to very large neural networks. It should be noted that the network size is an important consideration, not as much as for image or text problems, but for applications where the generation of additional data points involves a significant cost.

[0018] Alternatively, a single deterministic neural network can be used to evaluate the uncertainty in deep neural networks. In the first group of solutions, the uncertainty can be estimated using a second neural network, i.e. using a metric of distance to the training set or a statistical hotelling of the latent space. None of the above methods affect already trained models. The problem of indefinability is fully visible in the concept of measuring the distance to the training set. In this method, the distance is assumed to be representative of the reliability of the prediction. However, the distance generated is derived from the selected definition and can vary greatly between definitions. In other words, the fact that the distance is close in one definition does not mean that the distance is close in all definitions. Moreover, the distance observed in one trained model may not even correspond to the distance observed in the second model. Thus, in a recursive manner, additional systems that prove the results of the previous system are required. In summary, the search for intra-domain evaluation, reliability metrics or explainable AI is limited by the theorem of indefinability. Summary of the Invention [Problem to be solved by the invention]

[0019] The present invention aims to address all or part of these shortcomings. [Means for solving the problem]

[0020] According to a first aspect, the present invention provides a system for training an ensemble neural network device, comprising one or more computer processors and one or more computer readable media operatively coupled to the one or more computer processors, the one or more computer readable media, when executed by the one or more computer processors, causing the one or more computer processors to perform the following steps: - providing a sample data set, the sample data set including at least one input set and at least one output set associated with the input set, to a neural network device, the neural network device including an ensemble of neural network devices configured to provide independent predictions based on the sample data; - operating a neural network device on the sample data set; obtaining a trained neural network device configured to provide an output; 1. A system storing instructions for executing the neural network device further comprises at least two independent activation functions, at least two of the independent activation functions representing a statistical distribution of a plurality of independent predictions, and the neural network device is configured to provide at least one output for the at least two of the independent activation functions; - the operating step further comprises operating each neural network device of the ensemble to provide an ensemble of outputs, the neural network devices being trained to minimize values ​​representing at least two of said independent activation functions; Regarding the system.

[0021] Such an embodiment allows for significantly higher prediction stability, reliability, and improved training speed and overall performance, and also provides a variance metric that expresses the model uncertainty. Thus, the embodiment allows for resource savings in terms of computation time or power, and in terms of model complexity. Typically, current approaches require the use of multiple models and multiple iterations to obtain a reliable predictive model.

[0022] In this embodiment, for example, the mean of the distribution around the target is trained to be minimized, and the variance of the distribution is trained to be minimized to a value close to or equal to zero, thereby obtaining the smallest possible variance in the trained ensemble neural network device in a single call.

[0023] In the prior art, the gradient of the loss function is sometimes modified as a function of variance, but this is based on the unverifiable hope that the loss function reduces the problem of scattering prediction, which consists of simultaneously solving at least two equations in the loss. This results in a multi-objective optimization problem. According to logical rules, a compromise must be found between these at least two equations to be minimized, which is more complicated than if there was only one objective function to be minimized. In a particular embodiment, the neural network device obtained in the obtaining step is additionally configured to provide a value that represents the variance of the output.

[0024] Such an embodiment allows for the explainability of larger models.

[0025] In certain embodiments, at least two of the activation functions are: - the average of the statistical distribution of multiple independent predictions, and - Variance of the statistical distribution of multiple independent predictions It expresses the following.

[0026] In certain embodiments, the neural network device further includes a layer configured to add a simulation of an output generated by using the learned distribution of the plurality of independent outputs as the trained functions of at least two of the at least two independent activation functions.

[0027] Such an embodiment allows the output to be augmented so that it represents the distribution when the initial input is too small.

[0028] According to a second aspect, the present invention provides a computer-implemented method for training a neural network device, comprising the steps of: - providing a sample data set, the sample data set including at least one input set and at least one output set associated with the input set, to a neural network device, the neural network device including an ensemble of neural network devices configured to provide independent predictions based on the sample data; - operating a neural network device on the sample data set; obtaining a trained neural network device configured to provide an output; Including, - operating the neural network device, the neural network device including at least two independent activation functions, at least two of the independent activation functions representing a statistical distribution of a plurality of independent predictions, the neural network device configured to provide at least one output for at least two of the independent activation functions; - the operating step further comprises operating each neural network device of the ensemble to provide an ensemble of outputs, the neural network devices being trained to minimize values ​​representing at least two of said independent activation functions; It relates to a computer-implemented method.

[0029] The method that is the subject of the present invention offers the same advantages as the system that is the subject of the present invention.

[0030] According to a third aspect, the invention relates to a computer-implemented neural network device obtained by the computer-implemented method that is the subject of the invention.

[0031] The computer-implemented neural network device that is the subject of the present invention offers the same advantages as the system that is the subject of the present invention.

[0032] According to a fourth aspect, the invention relates to a computer program product comprising instructions for carrying out each of the steps of the method that is the subject of the invention, when the program product is executed on a computer.

[0033] The computer program product that is the subject of the present invention offers the same advantages as the system that is the subject of the present invention.

[0034] According to a fifth aspect, the invention relates to a computer-readable medium having stored thereon instructions for carrying out the steps of the method that is the subject of the invention, when the instructions are executed on a computer.

[0035] The computer readable medium that is the subject of the present invention offers the same advantages as the system that is the subject of the present invention.

[0036] According to a sixth aspect, the present invention provides a computer implemented method for predicting a physical, chemical, medical, sensory or pharmaceutical property of a flavour, fragrance or pharmaceutical ingredient, the computer implemented method comprising: - training, by a computing device, a neural network device according to the method of the present invention, wherein the example data set comprises: as input a composition of flavour, fragrance or pharmaceutical ingredients, and as output, at least one physical, chemical, medical, sensory or pharmaceutical property, one of which is the molecular weight of the composition; The steps represent - inputting, on a computer interface, a digital identifier of at least one flavour, fragrance or pharmaceutical ingredient, wherein the resulting input corresponds to a composition of the flavour, fragrance or pharmaceutical ingredient; - operating a trained neural network device with a computing device; - providing, on a computer interface, at least one physical, chemical, medical, sensory or pharmaceutical property output by the trained neural network device for the composition; The present invention relates to a computer-implemented method,

[0037] According to a seventh aspect, the present invention provides a computer implemented method for predicting an expression category in an image, the method comprising: - training, by a computing device, a neural network device according to the method of the present invention, wherein the example data set comprises: - an image as input, and - as output, at least one representation category in the input image and - inputting at least one image on a computer interface; - operating a trained neural network device with a computing device; - providing, on a computer interface, at least one representation category for the composition output by the trained neural network device; The present invention relates to a computer-implemented method,

[0038] Such a definition allows for image-based predictions, such as those used in medical imaging or cell activation.

[0039] Other advantages, objects and particular features of the present invention will become apparent from the following non-exhaustive description of at least one specific embodiment of the invention in conjunction with the accompanying drawings. [Brief description of the drawings]

[0040] [Figure 1]FIG. 1 illustrates a schematic diagram of a generalized representation of neural network device components. [Diagram 2] FIG. 1 shows diagrammatically a first particular sequence of steps of the system that is the subject of the present invention; [Diagram 3] FIG. 2 shows diagrammatically a second particular sequence of steps of the system that is the subject of the invention. [Figure 4] FIG. 2 shows a schematic diagram of a first particular sequence of the method for the system that is the subject of the invention; [Diagram 5] FIG. 1 is a schematic diagram of a first specific sequence of a method for predicting the physical, chemical, medical, sensory or pharmaceutical properties of a composition of fragrance or flavor ingredients of the present invention. [Figure 6] 1 shows a schematic representation of a first particular sequence of the method for predicting the representation category of an image that is the subject of the invention; [Figure 7] 1 shows a schematic representation of a computer architecture adapted to carry out the method that is the subject of the present invention; [Figure 8] FIG. 1 shows a schematic of two compared specific architectures used to predict molecular weights of molecules. [Figure 9] FIG. 1 shows performance results for two specific architectures. [Figure 10] FIG. 1 shows performance results for two specific architectures. [Figure 11] FIG. 1 shows performance results for two specific architectures. [Figure 12] FIG. 1 shows performance results for two specific architectures. [Figure 13] FIG. 1 shows performance results for two specific architectures. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0041] The description herein is not exhaustive, as each feature of an embodiment may be advantageously combined with any other feature of any other embodiment.

[0042] Various inventive concepts may be implemented as one or more methods, examples of which are presented. The acts performed as part of a method may be ordered in any suitable manner. As such, embodiments may be constructed in which acts are performed in an order different from that shown, which may include performing some acts simultaneously even though in the illustrated embodiments they are shown as sequential acts.

[0043] The indefinite article "a" or "an" as used in the specification and claims, unless otherwise specified, should be understood to mean "at least one."

[0044] The term "and / or" as used herein and in the claims should be understood to mean "either or both" of the elements connected by the term, i.e., elements that are conjunctively present in some cases and alternatively present in other cases. Multiple elements listed with "and / or" should be interpreted in the same manner, i.e., as "one or more" of the elements so combined. Other elements other than the elements specifically identified in the "and / or" clause may also be present as an option, whether related or unrelated to the elements specifically identified. Thus, as a non-limiting example, when used with an open-ended term such as "comprising," a reference to "A and / or B" may, in one embodiment, refer to only A (optionally including elements other than B), in another embodiment to only B (optionally including elements other than A), in yet another embodiment to both A and B (optionally including other elements), and so forth.

[0045] As used herein and in the claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as inclusive, i.e., including at least one element, and including more than one element of a plurality or list of elements, and including additional items not listed as an option. Conversely, words with clear indications, such as "only one of" or "exactly one of," or "consisting of," when used in the claims, refer to the inclusion of only exactly one element of a plurality or list of elements. In general, the word "or" as used herein should be interpreted as indicating exclusive alternatives (i.e., "one or the other, but not both") only when accompanied by a term indicating exclusivity, such as "any of," "one of," "only one of," or "exactly one of." When used in the claims, "consisting essentially of" has its ordinary meaning as used in the field of patent law.

[0046] As used herein and in the claims, the phrase "at least one" in reference to a list of one or more elements should be understood to mean at least one element selected from any one or more elements in the list of elements, but not necessarily including at least one of each of the elements specifically listed in the list of elements, and not excluding any combination of elements in the list of elements. By this definition, each element other than the element specifically identified in the list of elements referred to by the phrase "at least one" may be present as an option, whether related or unrelated to the element specifically identified. Thus, as a non-limiting example, "at least one of A and B" (or the equivalent "at least one of A or B" or the equivalent "at least one of A and / or B") can refer in one embodiment to at least one, optionally including two or more As, and no B (and optionally including elements other than B); in another embodiment to at least one, optionally including two or more Bs, and no A (and optionally including elements other than A); in yet another embodiment to at least one, optionally including two or more As, and at least one, optionally including two or more Bs (and optionally including other elements);

[0047] In the claims and this specification, all transitional phrases such as "comprising," "including," "carrying," "having," "containing," "attached to," "holding," "consisting of," and the like, shall be understood to be open-ended, i.e., meaning including, but not limited to, the recited elements. Additionally, the transitional phrases "consisting of" and "consisting essentially of" shall be intended to be closed-ended or semi-closed-ended, respectively.

[0048] It should be noted that the drawings are not drawn to scale.

[0049] The term "volatile ingredient" as used herein preferably refers to any ingredient that exhibits flavoring or fragrance-imparting capabilities. The terms "compound" or "ingredient" refer to the same item as "volatile ingredient." An ingredient may be formed from one or more chemical molecules.

[0050] The term composition refers to a liquid, solid or gaseous preparation of at least one fragrance or flavor ingredient.

[0051] As used herein, "flavor" refers to the olfactory perception resulting from the sum of the activation, enhancement, and (if present) inhibition of odorant receptors and the activation of taste buds containing taste receptor cells via orthonasal and retronasal olfaction by at least one volatile component. Thus, by way of example and not intended to limit the scope of the present disclosure, "flavor" is obtained from the sum of a first volatile component that activates odorant receptors or taste buds associated with coconut tonality, a second volatile component that activates odorant receptors or taste buds associated with celery tonality, and a third volatile component that inhibits odorant receptors or taste buds associated with hay tonality.

[0052] As used herein, "fragrance" refers to the olfactory perception resulting from the sum of the activation, enhancement, and (if present) inhibition of odorant receptors by at least one volatile component. Thus, by way of example and not intended to limit the scope of the present disclosure, a "fragrance" is obtained from the sum of a first volatile component that activates odorant receptors associated with coconut tonality, a second volatile component that activates odorant receptors associated with celery tonality, and a third volatile component that inhibits odorant receptors associated with hay tonality.

[0053] The term "input means" as used herein is for example a keyboard, a mouse and / or a touch screen adapted to interact with the computing system in a manner that allows collection of user input. In a variant, the input means is logical in nature, for example a network port of the computing system configured to receive electronically transmitted input commands. Such input means may be associated with a GUI (Graphics User Interface) or an API (Application Programming Interface) that is presented to the user. According to another variant, the input means may be a sensor configured to measure a specific physical parameter related to the intended use case.

[0054] As used herein, the term "computing system" or "computer system" refers to any electronic calculation device, whether centralized or distributed, capable of receiving numerical input and providing digital and / or analog numerical output by or to any type of interface. Typically, a computing system refers to either a computer running software with access to data storage, or a client-server architecture where data and / or calculations are performed on the server side while the client side acts as an interface.

[0055] The term "digital identifier" as used herein refers to any computerized identifier, e.g., an identifier used in a computer database, to represent a physical object, e.g., a flavoring ingredient. A digital identifier can also refer to a label that represents the name, chemical structure, or internal reference of the flavoring ingredient.

[0056] The term "human response" as used herein refers to any physical behavior elicited by confronting a human with a composition, which may be broadly defined, such as an appreciation or aversion to the composition, or may be more specifically described, such as, for example, facial expressions or body movements when confronted with the composition.

[0057] As used herein, the term "embodied" is intended to have an existence outside the digital environment of the present invention. "Embodied" can mean, for example, that it is readily found in nature or that it is synthesized in a laboratory or chemical plant. In either case, the embodied composition exhibits tangible reality. The terms "compounded" or "compounded" refer to the act of embodying a composition, whether by extraction and blending of ingredients or by synthesis and blending of ingredients.

[0058] The term "activation function" as used herein defines how in a neural network a weighted sum of inputs is transformed into an output from one or more nodes in a layer of the network. The activation function can be defined by a layer in the network or by an arithmetic solution to a loss function.

[0059] The embodiments disclosed below are presented in a general manner.

[0060] 2 shows a particular embodiment of a system 200 that is the subject of the present invention. The system 200 for training an ensemble neural network device comprises one or more computer processors and one or more computer readable media operatively coupled to the one or more computer processors, the one or more computer readable media being operable to, when executed by, cause the one or more computer processors to perform the following steps: - providing 205 a sample data set comprising at least one input set 220 and at least one output set 225 associated with the input set to a neural network device comprising an ensemble 230 of neural network devices configured to provide independent predictions based on the sample data; - a step 210 of operating a neural network device on the example data set; - obtaining 215 a trained neural network device configured to provide an output; wherein: the neural network device further comprises at least two independent activation functions, at least two of the independent activation functions representing a statistical distribution of a plurality of independent predictions, the neural network device being configured to provide at least one output 235, 236 for the at least two of the independent activation functions; the operating step further comprises operating each neural network device of the ensemble to provide an ensemble of outputs, the neural network devices being trained to minimize values ​​representing at least two of said independent activation functions.

[0061] The system 200 itself may be formed from any combination of means for performing the characteristic steps carried out by a computer processor.

[0062] The providing step 205 can be performed via a computer interface, such as an API or any other digital input means. The providing step 205 can be initiated manually or automatically. The specimen dataset can be assembled from a larger specimen dataset manually on a computer interface or automatically by a computing system.

[0063] The sample data is, for example, - at least one digital identifier of at least one fragrance, flavor or pharmaceutical ingredient forming the composition, which is optionally associated with the composition identifier; and Molecular weight of the composition may include.

[0064] Such a representative data set can be obtained by formulating the compositions and mathematically adding up the theoretical weights of each atom to obtain the molecular weights of the compositions in the representative set.

[0065] In other variations, the sample data may be, for example, -At least one image, and - for at least one of said images, the category in the list of possible categories that the image represents (e.g. "airplane", "vehicle", "bird", "cat", "deer", "dog", "frog", "horse", "ship", "truck"); Such categories may also be referred to as tags, labels, invocations or classes of expressions.

[0066] The operating step 210 can be performed, for example, by a computer program executed on a computing system. The ensemble neural network device is configured to train on input data during the operating step 210. Each neural network of the ensemble neural network device configures coefficients of a layer of artificial neurons during the operating step 210 to provide an output forming an output distribution. Values ​​of statistical parameters describing the distribution are obtained and can be used in the activation function to be minimized.

[0067] In certain embodiments, at least two of the activation functions are: - the average of the statistical distribution of multiple independent predictions, - the variance of the statistical distribution of multiple independent forecasts, and As an optional measure, - skew in the statistical distribution of multiple independent predictions, and / or -Kurtosis of the statistical distribution of multiple independent forecasts With an additional activation function that represents It expresses the following.

[0068] The obtaining step 215 can be performed via a computer interface, such as an API or any other digital output system. The obtained trained ensemble neural network device can be stored in a data storage, such as a hard disk drive or a database.

[0069] In certain embodiments, the neural network device obtained during the obtaining step 215 is additionally configured to provide at least one value representing the statistical variance of the output.

[0070] In FIG. 3, a particular embodiment of a system 200 that is the subject of the present invention is shown.

[0071] In this embodiment, the neural network device further includes a layer 240 configured to add a simulation 245 of an output generated by using the learned distribution of the multiple independent outputs as the trained functions of at least two of the at least two independent activation functions.

[0072] Such an embodiment may correspond, for example, to a Gaussian expansion of the output provided by the neural network device, based on the mean and variance of the output.

[0073] FIG. 4 illustrates generally a particular sequence of steps of a computer-implemented method 300 for training a neural network device, the method comprising: - providing 305 a sample data set comprising at least one input set and at least one output set associated with the input set to a neural network device comprising an ensemble of neural network devices configured to provide independent predictions based on the sample data; - a step 310 of operating a neural network device on the example data set; - obtaining a trained neural network device 315 configured to provide an output; Including, - operating the neural network device, the neural network device further comprising at least two independent activation functions, at least two of the independent activation functions representing a statistical distribution of a plurality of independent predictions, and the neural network device configured to provide at least one output for at least two of the independent activation functions; the operating step further comprises a step 320 of operating each neural network device of the ensemble to provide an ensemble of outputs, the neural network devices being trained to minimize values ​​representing at least two of said independent activation functions.

[0074] A providing step 305, operating steps 310 and 320, and an obtaining step 315 are disclosed with reference to the corresponding steps of the system 200 that is the subject of the present invention shown in FIGS.

[0075] The invention relates to a computer-implemented neural network device, characterized in that the neural network device is obtained by a computer-implemented method 300 as claimed in claim 5.

[0076] The present invention relates to a computer program product comprising instructions for performing the steps of the method 300 shown in FIG. 4 when executed on a computer.

[0077] The present invention relates to a computer readable medium having stored thereon instructions, when executed on a computer, for performing the steps of the method 300 shown in FIG.

[0078] In Fig. 5, a particular sequence of steps of the method 400 that is the subject of the present invention is shown in schematic form. The computer-implemented method 400 for predicting the physical, chemical, medical, sensory or pharmaceutical properties of flavor, fragrance or pharmaceutical ingredients (such as that disclosed in the publication Molecular descriptors for chemoinfomatics, Roberto Todeschini, Viviana Consonni second, revised and enlarged edition, Wiley et al.) comprises: - training 405, by a computing device, a neural network device according to the method 300 shown in FIG. 4, where the example data set comprises: as input a composition of flavour, fragrance or pharmaceutical ingredients, and - as output, at least one physical, chemical, medical, sensory or pharmaceutical property, one of which is the molecular weight of the composition; Step 405, which represents - inputting 410, on a computer interface, a digital identifier of at least one flavour, fragrance or pharmaceutical ingredient, where the input obtained corresponds to a composition of flavour, fragrance or pharmaceutical ingredients; -operating 415 the trained neural network device with the computing device; - providing 420, on a computer interface, at least one physical, chemical, medical, sensory or pharmaceutical property output by the trained neural network device for the composition; Includes.

[0079] The method 400 is the subject of the present invention as one embodiment of the subject system 200 disclosed with respect to FIG.

[0080] Thus, the molecular weight (MW) of the molecule can be predicted. The molecular weight is calculated by summing the atomic weights for all atoms in the molecule. By using such a simple target, it is possible to evaluate whether a given architecture can extract meaningful molecular information from any data size, considering that the molecular weight can be calculated for any given molecule proposed. The main advantages of such an approach include the fact that it has an accurate metric that shows the absence of variance in the measurements. In contrast to molecular weight, the experimentally measured target is in fact not just a sum of atomic weights, but a complex function of the conditions used. If the molecular weight can be predicted accurately, it is at least possible to verify that the model correctly extracted chemical knowledge from the data, taking into account the disadvantages of uncertainty listed above. Now, the evaluation of such chemical knowledge extraction is not straightforward if the model is trained only on the experimental target.

[0081] Further considerations and embodiments are disclosed below.

[0082] Regarding chemical properties, we present results on the prediction of molecular weight in the neural network of the present invention. Although the task of predicting molecular weight seems clear, the prediction of molecular weight has two main advantages: first, the target is a precise value with a minimum to zero value variance. Thus, in experiments, we can evaluate the results eliminating the variance of the data as an explanation for the results. Second, the prediction clearly tells us whether the neural network was able to perform the correct chemical extraction.

[0083] The comparison here is performed to predict molecular weight by a single deterministic neural network (SDNN) and an ensemble neural network trained using mean and variance (MSENN). The model is trained using a recurrent neural network, typically used in natural language processing. The input is defined by tokenized vectors that define the atoms and bonds of the molecule, very similar to the tokenized input typically used in natural language processing. An example of such a format is a SMILES string. The results are calculated on an internal dataset of 9979 molecules typically found in natural plants used for olfactory, gustatory and medical properties with molecular weights below 450.

[0084] The first result shows that the reproducibility of SDNN is limited. First, it can be seen that the performance of SDNN varies widely. In fact, large differences can be seen in the RMSE of the performance of both the training set 905 and the test set 910, even though the variance on the data is limited. Note that all networks are trained starting from the same initial weights. Second, it is observed that the performance strongly depends on the split of data used. In fact, it can be observed that the test performance is overly optimistic for some data splits, well balanced for some points, and overly pessimistic for most splits. Based on the varying results, it can already be defined that the expected performance can vary significantly. In other words, the performance obtained for one test set does not indicate knowledge of the performance for another test set. Even if it is expected that the performance on future unseen data may show the same performance variance, the exact performance strongly depends on the evaluated selection. In fact, the sample size and sample bias for the future selected data set strongly influence the performance.

[0085] The second result compares the performance of our ensemble neural network against that of an SDNN with the same training-test split between the two networks. First, it can be seen that, with one exception, the performance 1005 on the training set is stable, with RMSE between 0.5 and 0.6. The distribution of observed RMSE values ​​for the training and test sets shows a clear downshift 1010 for the ensemble model compared to the single model. Second, it can be seen that the performance on the test set is significantly improved compared to the performance of the single network. As mentioned before, performance varies depending on the split used. In fact, while the training performance is quite robust for our ensemble neural network, the range of test performance 1015 is quite large. These results again suggest that performance on one test is not indicative of performance on another test. As mentioned above, a similar variance should be expected when examining performance on future unseen data. Third, we can see that when using the ensemble neural network, the neural network reduces the RMSE by 15-30%. This reduction is observed for both the training and test sets in data1020 and data1025.

[0086] In summary, ensemble neural networks that are actively trained based on the mean and variance show better performance compared to single deterministic neural networks.

[0087] While normal distribution training of the mean and variance can be used, the method can also be used with other statistical distributions.

[0088] In Figure 8, two neural network devices are shown trained to predict the molecular weight of a molecule. For both networks, the input is a set of multiple NLP-compatible representations defining the same molecule as shown for the diethyl ether molecule. The embedding layer is a discrete integer to continuous vector transformation. The RNN layer applies a time sequence analysis. Examples of layers are GRU or LSTM. Multi-sequence attention defines a pooling mechanism to combine the knowledge obtained from all the processed input sequences. MLP defines a multi-layer perceptron composed of multiple fully connected layers that reduce the size of the activations by the “selu” activation function, although other activation functions such as “luckyrelu”, “relu”, and “elu” can also be used. MW defines a layer that predicts the target value, i.e., A) An architecture for a single deterministic neural network, where the network is trained based on expectations; B) Normal distribution N(μ,σ 2 The architecture of the inventive ensemble consists of an ensemble of n MLPs that generate MW as . The model is trained to minimize the expectation and the mean and variance relative to zero, respectively; It is.

[0089] In Figure 9, the performance of the molecular weight prediction by a single deterministic network is shown. The values ​​shown are the root mean square error (RMSE) and the correlation coefficient (R2). The black lines indicate the case where the RMSE or R2 between training and testing is equal. That is, A) Results shown for RMSE(test) vs. RMSE(training); B) Results shown for R2(test) vs. R2(training); It is.

[0090] In Fig. 10-12, the performance of the prediction of molecules by the ensemble neural network device of the present invention is indicated by "distributed ensemble", while the single deterministic neural network is indicated by "single". Reference example 1005 shows a comparison of RMSE(distributed ensemble) vs. RMSE(single) on the training set for each split. Reference example 1020 shows a comparison of RMSE(distributed ensemble) vs. RMSE(single) on the test set for each split. Reference example 1030 shows a histogram of RMSE for the training set in single (black) and distributed ensemble (gray). Reference example 1010 shows a histogram of RMSE for the test set in single (black) and distributed ensemble (gray). Reference example 1025 shows an error reduction analysis of the performance of the training set calculated as RMSE(distributed ensemble):RMSE(single), i.e., a value less than 1.0 indicates an improvement. Example 1035 shows an error reduction analysis of the test set performance calculated as RMSE(distributed ensemble):RMSE(single), i.e., values ​​less than 1.0 indicate improvement. Example 1015 shows the RMSE(test) vs. RMSE(training) performance for the distributed ensemble of the ensemble neural network of the present invention.

[0091] In Figure 13, a comparison between the average ensemble and the preset variance ensemble is shown. The predictions of the classical average ensemble using the same split can be compared with the variance ensemble of the present invention, which considers all molecules with a molecular weight (MW) below 450. The variance ensemble proposed here predicts MW with a significant improvement in the accuracy or uncertainty of the prediction. In the average ensemble, the model reports a standard deviation of 4 (95% cutoff), corresponding to a difference of 8 hydrogen atoms. However, the variance ensemble model shows a standard deviation of 0.8 (95%), corresponding to a difference of less than 2 hydrogens. The known number of hydrogens involved defines important information about the saturation level in the molecule.

[0092] In fact, Figure 13 shows a comparison of the mean and variance ensembles for predicting molecular weight (MW). A) Plot of variance of standard deviation (variance ensemble, y-axis) versus standard deviation (mean ensemble, x-axis); B) Reported value distribution of standard deviations for the variance ensemble (black) and mean ensemble (gray) inferences; C) Histogram showing the improvement in terms of reported standard deviation, which is the standard deviation of the mean ensemble divided by the standard deviation of the variance ensemble; It is.

[0093] 6, a particular sequence of steps of the method 500 that is the subject of the present invention is shown in schematic form. The computer-implemented method 500 for predicting an expression category in an image comprises: - training 505, by a computing device, a neural network device according to the method 300 shown in FIG. 4, wherein the example data set comprises: - an image as input, and - as output, at least one representation category in the input image Step 505, including: - a step 510 of inputting at least one image on a computer interface; -operating 515 the trained neural network device with the computing device; - providing 520, on a computer interface, at least one representation category output by the trained neural network device for the composition; Includes.

[0094] The subject method 500 is directed to one embodiment of the subject system 200 disclosed with respect to FIG.

[0095] Further considerations and embodiments are disclosed below.

[0096] Neural networks are an emerging trend that is being deployed for widespread applications. Based on the large amount of images, neural networks have been widely adopted for image problems. In many cases, image learning of neural networks can be easily solved using a single deterministic neural network. The main drawback of a single deterministic neural network is that it does not convey the uncertainty of the predictive model. The second drawback of such networks is that most single networks are not robust and are highly vulnerable to data perturbations, more colloquially known as adversarial examples. In summary, model uncertainty assessment is recognized as one of the key areas that are yet to be solved.

[0097] Recently, evidence deep learning has been introduced to estimate the uncertainty of models for image classification. In this method, a single additional evidence layer is introduced to provide parameters for a preselected distribution. The variance equation for the selected distribution can then be applied to mathematically calculate the variance. According to the work of Sensoy et al. (https: / / arxiv.org / abs / 1806.01768), the Dirichlet distribution is used as a source of variance. This led to improved robustness against out-of-domain query detection and adversarial perturbations. The first major drawback of this method is introduced with the choice of distribution. Thus, the resulting variance is a bounded result for the system trained here. In a regressional approach, it can even be said that the solution of variance is equally subject to the same concerns. In fact, these new parameters also come from a new single deterministic neural network.

[0098] To improve these shortcomings, the present invention uses ensemble neural networks (ENNs). ENNs are introduced to improve the robustness of neural networks, but also to provide notions of model uncertainty. Examples of ENNs are test time average ensembles, bootstrapping ensembles, snapshot ensembles, dropout ensembles, average ensembles, mean variance ensembles, and ensembles trained using negative correlation learning.

[0099] Of this group, snapshot ensembles and dropout ensembles stand out, as they are typically computed using a single deterministic neural network. In snapshot ensembles, multiple weight configurations at different times are combined. The resulting variance is therefore a metric of time stability for the predicted points. In dropout ensembles, uncertainty is introduced by applying a dropout layer also at inference time. The generated variance is therefore a measure of the stability of the parameter subsampling. A drawback of both networks is that the prediction uncertainty is often underestimated. In snapshot ensembles, this is a consequence of time dependency. In dropout ensembles, variables can exist as multiple choices. These can be improved by applying neural networks with a very high dropout rate. However, this can have a significant impact on the size of the network.

[0100] A particular solution is provided by bootstrapping ensembles. In this type of network, an ensemble is created by training the same network with different data selections. The notion of the uncertainty of the generated model is then a metric or robustness to data subsampling. In this type of ensemble, the high density points are well supported and are not affected by the subsampling. The same concerns that arise for dropout networks arise for bootstrapping ensembles. It is expected that the uncertainty may be underestimated, since a small bootstrapping omission rate may result in repeated use of data points. The latter is particularly beneficial for points that arise from a high density rate in the data.

[0101] The group of mean ensembles and test-time mean ensembles defines a group of ensemble networks that are combined to generate the mean and variance for prediction. The mean ensemble is actively trained on the mean, whereas the test-time ensemble is an ensemble of independently trained networks. In the case of the mean network, the model is trained on the predicted right-side mean. The main drawback of training only on the mean is the fact that the variance of the submodels can vary significantly between predictions at different points. In the test-time ensemble, all models are trained independently. As a result, these networks may even exhibit the problem that the mean between the networks is not optimized as when the optimization is done on the mean ensemble counterpart.

[0102] The mean-variance ensemble and the lower-upper bound ensemble neural networks are trained using the data variance. The lower and upper bounds are the mean and variance of the variance, i.e. the lower and upper bounds are calculated as mean-variance and mean+variance, respectively. In these networks, the network is trained using the data variance. The main drawback of this approach is that the variance does not provide any conclusions on the uncertainty of the model. Moreover, the data variance itself is not a property of the predicted points, but rather a property of the observed variation in their measurements. In fact, the reported variance strongly depends on the number of measurements performed for each data point. The number of reported data points can vary significantly from point to point.

[0103] The above mentioned drawbacks are addressed by applying a strategy called negative correlation learning. In this approach, the losses are typically modified to take into account the diversity in the signal. Examples of proposed training mechanisms are the use of coupling terms or the use of KL divergence. The method has been widely evaluated and it has been observed that the performance of the base learner varies strongly. It has been reported that the method is usually beneficial for small capacity base learners but detrimental for large capacity base learners. In summary, the use of NCL in ensembles requires hard fine-tuning optimization to obtain the best possible results.

[0104] When training a single deterministic neural network, it is generally not possible to determine whether the initialization of such a network will give the best results. Moreover, it is not possible to accurately determine whether the model has developed a bias for the particular subset of data used. In extreme cases, it can be observed that the model may fail to provide answers to some of the questions asked, i.e., it fails to correctly predict some points.

[0105] In a particular embodiment herein, an ensemble neural network device is trained on both the mean and the variance to establish communication between sub-models of the ensemble neural network. As a simplification of the training mechanism, a sampling mechanism is applied to the mean and variance generated in the ensemble, similar to the sampling mechanism used in a variable autoencoder (VAE).

[0106] Note that, unlike the system of the present invention, the VAE is a single deterministic neural network that uses independent layers of random distribution and becomes a generative neural network by applying a sampling mechanism.

[0107] In this work, the methodology is applied to image classification using the CIFAR-10 dataset, which asks the model to predict one class out of 10 possible classes for a set of images. Results are calculated for five different splits with a training size of 50000 images and a test set of 10000 images. Results are summarized by measuring the classification accuracy, i.e. the percentage of correct predictions.

[0108] We compared the performance of network sampling using the mean and variance of the ensemble, sampling using the full covariance of the ensemble, and sampling from independent variance layers occurring within the network. Note that the latter method is the same as the strategy used in VAE. In the following table, the three methods are referred to as diagonal, full covariance, and diagonal MLP, respectively. We compared our method with five existing solutions: 1) mean ensemble, 2) negative correlation learning, 3) single deterministic neural network, 4) dropout ensemble, and 5) bootstrapping ensemble. In the table, these solutions are identified by the mean ensemble, NCL, single deterministic NN, dropout ensemble, and bagging ensemble, respectively. The results reported are prediction accuracy.

[0109] Performance results based on the methodology tested [sorted by decreasing precision] Methodology Verification accuracy Complete covariance (invention) 83.1±0.3% Dropout ensemble 82.8±1.1% Diagonal (present invention) 82.4±0.2% Negative correlation learning NCL 81.7±0.4% Bagging ensemble 81.6±0.2% Mean ensemble 79.2±0.3% Single deterministic NN 77.0±0.5% Diagonal MLP (present invention) 76.0±0.5%.

[0110] The results in the above table show some clear results. First, all ensemble neural networks outperform single deterministic neural networks. In fact, single deterministic neural networks and diagonal MLPs perform significantly worse than the top six tested ensemble methods in the table. Second, for diagonal MLPs, it can be seen that the use of independent random variance layers is not beneficial to improve the results. Furthermore, the results show that the performance degradation of diagonal MLPs is statistically significant when compared to single deterministic networks. Third, it can be seen that classical mean ensemble (average), bootstrapping ensemble (bagging) and negative correlation learning NCL can all improve prediction accuracy. Fourth, it can be seen that the ensemble techniques of the present invention, full covariance and pairwise, perform significantly better. Fifth, of the reported ensemble methods, only the dropout ensemble performs with comparable accuracy. However, it is noted that the dropout ensemble shows a rather strong variance to the reported performance. Our ensemble methodologies, full covariance and diagonal, show 0.3% and 0.2% variance, respectively, while the dropout ensemble shows a significantly larger variance of 1.1%, demonstrating that our method is more robust than existing dropout methods.

[0111] In summary, we reach a consensus that the ensemble neural network with communication submodels outperforms previously reported ensemble neural networks in terms of repeatability of prediction accuracy.

[0112] 7 shows a schematic diagram of a computer architecture 600 on which the system 200, the subject of the present invention, can be implemented. The computer architecture 600 includes: - providing a sample data set, the sample data set including at least one input set and at least one output set associated with the input set, to a neural network device, the neural network device including an ensemble of neural network devices configured to provide independent predictions based on the sample data; - means for operating a neural network device based on a sample data set; - a means for obtaining a trained neural network device configured to provide an output; Equipped with the neural network device further comprises at least two independent activation functions, at least two of the independent activation functions representing a statistical distribution of a plurality of independent predictions, and the neural network device is configured to provide at least one output for the at least two of the independent activation functions; the means for operating further comprises means for operating each neural network device of the ensemble to provide an ensemble of outputs, the neural network devices being trained to minimize values ​​representing at least two of said independent activation functions.

Claims

1. A system (200) for training an ensemble neural network device, comprising: The system (200) includes one or more computer processors and one or more computer readable media operatively coupled to the one or more computer processors, the one or more computer readable media that, when executed by the one or more computer processors, cause the one or more computer processors to perform the following steps: - providing (205) a sample data set comprising at least one input set (220) and at least one output set (225) associated with said input set to a neural network device comprising an ensemble (230) of neural network devices configured to provide independent predictions based on the sample data; - operating (210) said neural network device on said representative data set; - obtaining (215) a trained neural network device configured to provide an output; A system (200) storing instructions for executing the neural network device further comprises at least two independent activation functions, at least two of the independent activation functions representing a statistical distribution of a plurality of independent predictions, the neural network device being configured to provide at least one output (235, 236) for at least two of the independent activation functions; - the operating step further comprises operating each neural network device of the ensemble to provide an ensemble of outputs, the neural network devices being trained to minimize values ​​representing at least two of the independent activation functions; A system (200).

2. The neural network device obtained in the obtaining step (215) is additionally configured to provide a value representing a variance of the output. The system (200) of claim 1.

3. At least two of the activation functions are - the average of the statistical distribution of said plurality of independent predictions, and - the variance of the statistical distribution of said plurality of independent forecasts It expresses, The system (200) of claim 1.

4. the neural network device further includes a layer (240) configured to add a simulation (245) of an output generated by using the learned distribution of a plurality of independent outputs as a trained function of at least two of the at least two independent activation functions; The system (200) of claim 1.

5. 1. A computer-implemented method (300) for training a neural network device, the computer-implemented method (300) comprising: - providing (305) a sample data set comprising at least one input set and at least one output set associated with said input set to a neural network device comprising an ensemble of neural network devices configured to provide independent predictions based on the sample data; - operating (310) said neural network device on said representative data set; - obtaining a trained neural network device (315) configured to provide an output; A computer-implemented method (300) comprising: - in the step of operating the neural network device, the neural network device further comprises at least two independent activation functions, at least two of the independent activation functions representing a statistical distribution of a plurality of independent predictions, and the neural network device is configured to provide at least one output for at least two of the independent activation functions; - the operating step further comprises the step of operating each neural network device of the ensemble to provide an ensemble of outputs (320), the neural network devices being trained to minimize values ​​representing at least two of the independent activation functions; 1. A computer-implemented method (300).

6. A computer-implemented neural network device, characterized in that it is obtained by the computer-implemented method (300) of claim 5.

7. A computer program product, comprising instructions for carrying out each step of the method (300) according to claim 5, when said computer program product is executed on a computer.

8. A computer readable medium having stored thereon instructions, when executed on a computer, for performing the steps of the method (300) of claim 5.

9. A computer-implemented method (400) for predicting a physical, chemical, medical, sensory or pharmaceutical property of a flavor, fragrance or pharmaceutical ingredient, said method (400) comprising: - training (405) a neural network device according to the method (300) of claim 5, by a computing device, the example data set comprising: - as input, a composition of flavour, fragrance or medicinal ingredients, and - as output, at least one physical, chemical, medical, sensory or pharmaceutical property, one of which is the molecular weight of the composition; A step (405) expressing - inputting (410) on a computer interface a digital identifier of at least one flavour, fragrance or pharmaceutical ingredient, the input obtained corresponding to a flavour, fragrance or pharmaceutical ingredient composition; - operating (415) the trained neural network device by a computing device; - providing (420) on a computer interface at least one physical, chemical, medical, sensory or pharmaceutical property output by the trained neural network device for said composition; 20. A computer-implemented method (400) comprising:

10. A computer-implemented method (500) for predicting an expression category in an image, the computer-implemented method (500) comprising: - training (505) a neural network device according to the method (300) of claim 5, by a computing device, the example data set comprising: - an image as input, and - as output, at least one representation category in the input image A step (505) including: - inputting (510) at least one image on a computer interface; - operating (515) the trained neural network device by a computing device; - providing (520) on a computer interface at least one representation category output by the trained neural network device for the composition; 20. A computer-implemented method (500) comprising:

Images