Controlling the production of a product

Abstract

A method (1) of providing a control data set for controlling the production of a product comprises: receiving (10), by a computing device (2), an input data set, the input data set comprising a first target property value related to a first product property of the product, and a second target property value related to a second product property of the product; feeding (12), by the computing device (2), the input data set into a model to obtain, from the model, an output data set, the output data set including a plurality of results: determining (14), by the computing device (2), for each of the plurality of results, an associated quality score; selecting (18), by the computing device (2), one or more of the plurality of results based on the associated quality scores; and deriving (19), by the computing device (2), said control data set from the one or more selected result / s.

Description

[0001]CONTROLLING THE PRODUCTION OF A PRODUCT TECHNICAL FIELD This specification refers to embodiments of a method of providing a control data set for controlling the production of a product, e.g., a chemical product, to embodiments of a corresponding computing device, toembodiments of a method of producing a product and to embodiments of a system of producing a product.BACKGROUNDProducing a product, e.g., a chemical product, may occur in safety-critical environment, meaning that theproduct shall with a very high likelihood exhibit its designated target properties, such as, for example,weight, purity, lifetime, dimension, taste, design, medical effect, etc. Thus, control data based on which theproduction of the product is controlled must be chosen such that said high likelihood is indeed obtained.The abstract of the article “Multi-criteria optimization in chemical process design and decision support by navigation on Pareto sets”, by M. Bortz et al., Elsevier, Computers and Chemical Engineering 60 (2014), pg.354-363, states: “Designing chemical processes is a multi-criteria optimization problem with conflicting objectives. It can efficiently be solved using Pareto sets. These sets contain all solutions for which an improvement in any objective can only be achieved by accepting a decline in at least one other objective”.The article presents an algorithm to determine Pareto sets in a state-of-the-art steady-stateflow sheetsimulator. An approximation of predefined accuracy of the Pareto set, which can be convex or non-convex,is calculated. According to the article, a decision maker can then navigate interactively on the Pareto setand explore the different optimal solutions. His decision would, hence, be embedded in the knowledge ofthe entire Pareto set. SUMMARY The subject-matter of the independent claims is presented. Features of exemplary embodiments are defined in the dependent claims. According to first aspect, a method of providing a control data set for controlling the production of a product is presented. The method comprises: receiving, by a computing device, an input data set. The input data set comprises: a first target property value related to a first product property of the product, and a second target property value related to a second product property of the product; and data related to a first input variable and a second input variable, wherein first dependencies between the first input variable and the first and second product properties are different from second dependencies between the second input variable and the first and second product properties. The method further comprises feeding, by the computing device, the input data set into a model to obtain, from the model, an output data set, the outputdata set including a plurality of results. Each result indicates: a first result value of the first product property,a second result value of the second product property; and a third value of the first input variable and a fourth value of the second input variable, wherein said first, second, third and fourth value being associated with each other. The method further comprises determining, by the computing device, for each of theplurality of results, an associated quality score; selecting, by the computing device, one or more of theplurality of results based on the associated quality scores; and deriving, by the computing device, said control data set from the one or more selected result / s.According to second aspect, a computing device is configured to carry out the method according to the firstaspect.According to a third aspect, a method of producing a product is presented, wherein production of theproduct is carried out based on a control data set provided by a method according the first aspect.According to a fourth aspect, a system for producing a product is presented, wherein the system isconfigured to carry out the method of the third aspect. According to another aspect, this disclosure relates to a method (1) of providing a control data set for monitoring and / or controlling a production and / or a processing of a product, the method (1) comprising:- receiving (10), by a computing device (2), an input data set, the input data set comprising:o a first target property value related to and / or associated with a first product property of theproduct, and a second target property value related to and / or associated with a second product property of the product; odata associated with a first input variable and a second input variable, wherein one or morefirst dependencies between the first input variable and the first and second product properties are different from one or more second dependencies between the second input variable and the first and second product properties; and- feeding (12), by the computing device (2), the input data set into a model to obtain (13), from themodel, an output data set, the output data set including a plurality of results, each result indicating: oa first result value of the first product property, a second result value of the second productproperty; and oa third value of the first input variable and a fourth value of the second input variable; and- obtaining, in particular determining (14), by the computing device (2), for each of the plurality ofresults, an associated quality score;- selecting (18), by the computing device (2), one or more of the plurality of results based on theassociated quality scores; and- deriving (19), by the computing device (2), said control data set from the one or more selectedresult / s, optionally providing said control data set in particular for monitoring and / or controlling aproduction and / or a processing of a product.In accordance with some embodiments, it is proposed to determine a prediction uncertainty of arbitrarysurrogate model used to calculate a Pareto front as well as to incorporate this prediction uncertainty into areverse prediction task. Thus, not only arbitrary point predictions are determined, but also information aboutuncertainties caused, e.g., by data variance, model uncertainty, Pareto front approximation uncertainty.This may allow to provide a Pareto front navigation with a single descriptor that can be tuned to the risktolerance of a user.In the following, some exemplary features of exemplary embodiments will be described. Theses optional features can be combined with in an arbitrary manner to obtain even further embodiments, unless the description explicitly excludes specific combinations. The computing device may comprise a single computer or a distributed computer architecture, such as a server system, a computer / server farm or any other computer cluster.The product can be a chemical product, for example a product from one of the product groups lacquers,surfactants, paints or a cosmetics, such as creams. The input data set may be provided to the computing device via a wired and / or wireless data path. E.g., the input data set is provided by user using a user device, such as a terminal device. As described above, the input data set comprises a first target property value related to a first product property of the product, and a second target property value related to a second product property of the product. The input data set further data related to a first input variable and a second input variable, wherein first dependencies between the first input variable and the first and second product properties are different from second dependencies between the second input variable and the first and second product properties.For example, the first product property of the product is a chemical property, a physical property or abiological property, e.g., one of the following: weight, density, size, design, portion of a certain material, purity, taste, flammability, toxicity, acidity, reactivity (many types), and heat of combustion, medical effect etc.The second product property may be different from the first product property. E.g., the second productproperty of the product property of the product is a chemical property, a physical property or a biologicalproperty, e.g., one of the following: weight, density, size, design, portion of a certain material, purity, taste, flammability, toxicity, acidity, reactivity (many types), and heat of combustion, medical effect etc. In an embodiment, one or both of said target property values are absolute values. In another embodiment, one or both of said target property values define value ranges. In another embodiment, one or both of said target property values define that the respective product property is to be maximized or minimized or in another way to be optimized.The first input variable may relate, for example, to an educt (such as a chemical compound, e.g., an acid, abase, a soap or a solvent) based on which the product or is produced. Also the second input variable mayrelate, for example, to an educt based on which the product is produced. Additionally or alternatively, oneor more of said input variables may relate to parameters (such as temperature, feed rate, steering rate ortime) of the production process to be employed for the production of the product. Additionally oralternatively, one or more of said input variables may relate to one or more intermediate products that come into being during the production process employed for the production of the product. E.g., datarelated to more than two input variables are received, e.g., data related to up to 50 input variables arereceived. As explained above, the first dependencies between the first input variable and the first and second product properties are different from second dependencies between the second input variable and the first and second product properties. Accordingly, the first and second target property values and the data related to first and second input variables may define a multi-objective optimization with conflicting objectives, inaccordance with one or more embodiments.In a next step of the method, as described above, the input data set is fed into the model to obtain, from the model, the output data set. The output data set includes a plurality of results, wherein each results indicates: The first result value of the first product property, the second result value of the second product property, the third value of the first input variable and the fourth value of the second input variable. The four values are associated to each other, as they belong to one of the plurality of results. For example, the output data set includes at least a portion of a Pareto front (also called Pareto frontier or Pareto curve) that has been obtained based on the input data set and the model. The Pareto front may include at least a portion of all Pareto efficient solutions. In an embodiment, the model is generated and / or selected from a plurality of available models based onthe input data set. The model may be hosted by the computing device. Different input data sets mayrequire different models to obtain an appropriate output data set. For example, the model employed to obtain the output data set can be a multi-objective optimization model. E.g., the model includes one ormore of the following: a linear regression model, a partial least squares, PLS, model, a gaussian mixturemodel, a random forest model, a neural network model, e.g., a physics informed neuronal network model or the like. As also described below, the determination of the quality scores can be independent of the model chosen to obtain the output data set; i.e., said determination of the quality scores can be agnostic to the model applied for obtaining the output data set, in accordance with an embodiment. Simultaneously with or after the obtaining the output data set, the computing device determines, for each of the plurality of results, an associated quality score. In an embodiment, each quality score indicates howreliable the associated result is, e.g., based on a prediction interval, an uncertainty may be determined bythe computing device. E.g., the larger the interval, the larger is the uncertainty associated with the result. That is, in an embodiment, determining the associated quality scores includes an uncertainty estimation. E.g., a prediction interval estimation step is carried out after the model has provided the output data set, e.g., the Pareto points. E.g., based on each respective Pareto point, a prediction interval is calculated, sothat the final pareto front has confidence bounds. These confidence bounds can be evaluated automaticallyand / or evaluated by a user. For example, determining the quality scores is based on at least one of a first uncertainty value related to the input data set and a second uncertainty value related to the model. To this end, the input data set mayadditionally comprise experimental data about a measured or simulated relationship between (a) at leastone of the first input variable and the second input variable and (b) at least one of the first product property and the second product property. For example, based on the experimental data, the first uncertainty value related to the input data may be determined. In an embodiment, for each result, the quality score indicates a probability that the result is achieved within a certainty range, wherein said certainty range can be defined as part of the input data set. E.g., the input data set includes a certainty range, e.g., defined by a user, e.g., defining a range of 40% to 95%, meaningthat it is required that the result must be achieved with a certainty within the range of 40% to 95%. Therespective quality score in this case indicates, e.g., based on new observation / experiment carried out withthe step of determining the quality scores, the probability that the respective result is indeed with thespecified range. In another embodiment, for each result, the quality score indicates, based on theexperimental data, how many, e.g., in terms of percentage, cases of the determined values input variablesindeed yield the determined result values of the product properties. That is, whereas the determination ofthe quality scores may take into account the both the input data set and the output data set (including the results), the determination may occur independently from the model applied to obtain the output data set. Thus, the determination of the quality score may occur in a model-agnostic manner. Generally, uncertainty is a common term used in statistical modeling. E.g., two types of uncertainty can arise: aleatory uncertainty and epistemic uncertainty. Aleatory uncertainty (uncertainty arising from the data) arises from randomness and is inherent in any system. It is uncontrollable and unpredictable and is present in all natural phenomena, such as weather, sea levels, and stock markets. Epistemic uncertainty(uncertainty from the model) is caused by lack of knowledge or information. It is more controllable and canbe reduced through research and analysis. The technique described herein may address both uncertainty types depending on the provided input data set. As explained above, providing said experimental data, e.g., additional information about the measurement method and or providing measurement replicas allows to take the aleatory uncertainty into account when determining the associated quality scores. Determining the associated quality scores comprises, in an embodiment, applying a quantile regression method to the output data set. E.g., determining the associated quality scores excludes, in an embodiment, applying the method of least squares. For example, whereas the method of least squares estimates the conditional mean of the response variable (i.e., the first result values of the first product property and the second result values of the second product property) across values of the predictor variables (i.e., the third values of the first input variables and the fourth value of the second input variable), quantile regression can estimate the conditional median (or other quantiles) of the response variable. In an embodiment, the applied quantile regression method is independent of the model used for obtaining the output data set. E.g., the applied quantile regression method is model agnostic. For example, the quantile regression method may hence yield comparable quality scores even though different models mayhave been used for obtaining the output data set including the results.For example, the prediction intervals are used to provide an estimate of the uncertainty of the respectiveresult (e.g., said probability of being within the specified range, as described above) from the plurality ofresults. E.g., the prediction intervals consider both the uncertainty of the point estimate and the data scatteraround it. The prediction intervals may represent a range of values that are likely to contain the true valueof some response variable (product properties) for a single new observation based on specific values ofone or more predictor variables. For example, said prediction intervals are determined based on said quantile regression method. In the following, an embodiment of the method which includes the application of the quantile regression method will be exemplarily explained. Generally, the term “quantile regression” is herein used in the context the skilled person typically associates therewith. In a first step, the input data set is received. In the following example, the input data set comprises said data related to the first input variable and the second input variable, i.e., (i) data related to input variables. Furthermore, the input data set comprises (ii) said experimental data. In addition, the input data set comprises (iii) quantile range data indicative of a quantile range, e.g., said certainty range described above. For example, the quantile range is predefined. In another embodiment, the quantile range is selected, e.g., by the computing device and / or by a user device, e.g., based on corresponding input from a user of the user device. For example, the quantile range data specifies one or more quantiles, such two quantiles, e.g., 5% and 95%, or 10% and 80%. For example, the respective quantile defines a required probability for the results. E.g., the quantile range data, e.g., defined by a user, defines a range of 40% to 95%, meaning that it is required that each of the result must be achieved with a certainty within the range of 40% to 95%. The respective quality score in this case indicates, e.g., based on new observation / experiment carried out with the step of determining the quality scores, the probability that the respective result is indeed with the specified range. In a next step, a quantile regression model is generated. For example, the generated quantile regression model models the relationship between the input variables (e.g., independent predictor variables) and thespecified quantiles (also called percentiles) of the target properties (e.g., dependent variables).Generating the quantile regression model may occur in accordance with the following equation: ^^(^^) = ^^(^) + ^^(^)^^^ + … + ^^(^)^^^ , wherein^^(^^) is the ^ -th quantile of the response variable / target property ^^^ is the quantile^^(^) are the regression parameters / the coefficient to be estimated and^^^, …, ^^^ are predictor / input variables for the ^-th observation.Further, generating the quantile regression model may include decreasing the mean absolute deviation, MAD, for Quantile Regression until the MAD is below a predefined value according to the following equation: ^^^ =^∑^^ (^ − (^ (^) ^^^^^ ^ ^ ^ + ^^(^)^^^ + … + ^^(^)^^^)), wherein ^^is the function according to ^^(^) = ^ max(^, 0) + (1 − ^)max (−^, 0), wherein^ is the error of a single data point andmax() returns the largest value in the parentheses, e.g., u,0 or -u,0.Once the quantile regression model has been generated, the quantile range data and the data related to the input variables are fed into the quantile regression model. Then, the quality scores are obtained. As explained above, each result may be associated with at least one quality score. In an embodiment, the input data set may additionally comprise constraint data, the constraint dataindicating one or more inadmissible values related to at least one of the first input variable and the second input variable. For example, a constraint could define that a share of chemicals in a formulation should be 1 (one) in total. In an embodiment, the output data set does not present any results having valuescorresponding to the inadmissible values according the constraint(s).The method further comprises a selection step, e.g., executed by the computing device. Namely, one ormore of the plurality of results is / are selected based on the associated quality scores. Based on theselected result / s, the control data set for controlling production of the product is derived.The selection of the one or more results may occur entirely automatically or by additionally taking into account input from a user. In an embodiment of the method, the method further includes causing, by the computing device, rendering of a representation of the plurality of results and the associated quality scores. For example, the computing device itself includes a display for displaying the plurality of results and the associated quality scores. Additionally or alternatively, the presentation of the plurality of results and the associated quality scores may be provided by a user device. For example, the selection may be influenced based on input from a user which the user provided upon being confronted with the presentation of the plurality of results and the associated quality scores. The presentation may include at least one of a visual presentation, e.g.,including error bars to visualize the quality score, and / or an acoustic or haptic presentation.In a further embodiment, after the quality scores have been obtained, it may be determined, by the computing device, that none of the results has an associated quality score within a required range. Then, the computing device may automatically select another model to obtain the output data set. Based on the other model, the steps of obtaining the output data set and the step of determining the associated quality scores may be repeated. E.g., such iteration is repeated until the determined quality scores are within therequired range. In another embodiment, the model used to obtain the output data set is not changed. Forexample, if the desired result cannot be achieved by the model with the specified certainty range, then this result might not be achievable in reality, assuming that the applied model accurately represents reality. On the other hand, if a new observation contradicts the model, then the model may be updated or changed to a better suiting one. Determining the quality scores, e.g., by employing a quantile regression method, may allow for a reliable and accurate evaluation of the results which may, ultimately, increase the performance of formulations and saves resources otherwise used for experimentation. Based on the selection of the one or more results based on the associated quality score, the productionprocess of the product can be carried out such that the product exhibits the first and second target propertyvalues with a defined certainty. Thus, a request, e.g., by a customer, that a product must exhibit propertyvalues with a predefined likelihood can be automatically reflected based on the above-described techniqueof deriving the control data set for controlling the production of the product. Based on the associated qualityscores, only those results are chosen that comply with the customer request, in accordance with one or more embodiments. Accordingly, the above-described technique of deriving the control data set allows foran efficient and automatic adaptation of the control of the production of the product in accordance with therisk tolerance of the customer.In an embodiment, data related to may refer to data associated with. In particular, data related to and / orassociated with may comprise data representative of and / or data comprising. In an embodiment, one or more first and / or second dependencies may be represented by and / or maycomprise one or more mathematical equation(s). The one or more first and / or second dependencies maydescribe a relation between the first input variable and the first and second product properties and / or between the second input variable and the first and second product properties. Different first and / or second dependencies may comprise different relations between the first input variable and the first and second product properties and / or between the second input variable and the first and second product properties. In particular, different first and / or second dependencies may comprise different mathematical equations relating the first input variable and the first and second product properties and / or the second input variable and the first and second product properties.In an embodiment, the model may be configured to provide the output data set including a plurality ofresults in response to receiving the input data set. The data associated with the first input variable and / orthe second input variable may comprise one or more first proposed values associated with the first inputvariable and / or one or more second proposed values associated with the second input variable. Further,the data associated with the first and / or second input variable may comprise the one or more first and / orsecond proposed property values. The one or more first proposed input values and / or the one or moresecond proposed input values may be associated with one or more first and / or second proposed propertyvalues of the first and / or the second product property, in particular related to one or more first and / orsecond proposed property values of the first and / or the second product property via the one or more firstand / or second dependencies. The one or more first and / or second proposed input values and / or the one ormore proposed property values may be associated with each other, i.e. may be associated to a product,preferably a composition of the product. The one or more first and / or second result values and the third andthe fourth value may be associated to target product, i.e. a product other than the product associated withthe one or more first and / or second proposed input values. The model may be configured to obtain the firstand second result value and the third and the fourth value according to a data distribution indicated by theinput data set, in particular the data associated with the first and second input variable. Additionally or alternatively, the model may be configured to obtain the first and second result values closer to the first andsecond target property values than the one or more first and / or second proposed property values.In an embodiment, said first, second, third and fourth value being associated with each other may comprise said first, second, third and fourth value being associated with a product, in particular a target productand / or a composition of a product. Values associated with each other may be associated to the sameproduct and / or the same composition of the product. In an embodiment, obtaining, in particular determining the quality score may comprise obtaining the quality score from a quality and / or reliability model. The quality and / or reliability model may be configured toprovide a quality score associated with the plurality of results in response to receiving the plurality ofresults. The quality and / or reliability model may be obtained according to at least parts of the input data set,in particular according to the data associated with the first and / or second input variable.In an embodiment, deriving the control data set may comprise extracting at least a part of the selectedresults into a control data set. Providing the control data may comprise providing at least the part of theselected results. Selecting one or more of the plurality of results may comprise obtaining, in particularreceiving a selection of the one or more of the plurality of results via a user interface. The selection of theone or more of the plurality of results may be obtained, in particular received, in response to providing, in particular displaying, the plurality of results and the associated quality score(s) via the user interface.Displaying the plurality of results and the associated quality score(s) may comprise displaying at least apart of the results and displaying the quality score(s) associated with at least the part of the results.In an embodiment, obtaining the quality score may comprise obtaining a predefined quantile range and / orreceiving a quantile range data indicative of a quantile range, e.g. by specifying two or more quantiles (e.g.5 % and 95 % or smaller corridor where applicable) and providing the at least a part of the output data set,in particular the first and second result values, to the quality and / or reliability model and receiving thequality scores associated with the results from the quality and / or reliability model.Presented herein is also a computing device. The computing device is configured to carry out the methodaccording to one or more of the embodiments described above. Regarding embodiments of the computing device, it is hence referred to the above.Also presented herein is method of producing a product, wherein production of the product is carried outbased on a control data set provided by a method according to one or more of the embodiments described above. Further presented herein is a system for producing a product, wherein the system is configured to carry out the said production method based on the control data set. Those skilled in the art will recognize additional features and advantages upon reading the followingdetailed description, and upon viewing the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS The parts in the figures are not necessarily to scale, instead emphasis is being placed upon illustrating principles of the invention. Moreover, in the figures, like reference numerals designate corresponding parts / steps. In the drawings:Figs.1A-E schematically illustrate a portion of an exemplary method of providing a control data set forcontrolling the production of a product in accordance with one or more embodiments;Fig.2 schematically illustrates a portion of an exemplary method of providing a control data set forcontrolling the production of a product in accordance with one or more embodiments;Fig.3 schematically and exemplarily illustrates a Pareto front with confidence bounds and a Paretofront without confidence bounds;Fig.4 schematically illustrates a portion of an exemplary method of providing a control data set forcontrolling the production of a product in accordance with one or more embodiments;Fig.5 schematically and exemplarily illustrates a computing device and a user device in accordancewith one or more embodiments;Fig.6 schematically and exemplarily illustrates a system for producing a product in accordance withone or more embodiments; andFig.7 schematically illustrates a portion of an exemplary method of determining quality scores inaccordance with one or more embodiments. DETAILED DESCRIPTIONFigs.1A-E schematically illustrate a portion of an exemplary method of providing a control data set forcontrolling the production of a product in accordance with one or more embodiments, namely the step of receiving, by a computing device, an input data set, and the step of providing, by the computing device, an output data set.The input data set is in this example provided by a user employing a user device. In Fig.5, both thecomputing device 2 and the user device 8 are schematically illustrated. Both devices 2 and 8 comprise arespective memory 21 / 81 storing instructions for a respective processor 22 / 82 and a respective transceiver23 / 83 to receive and transmit data. The exact configuration of these devices 2 and 8 are presently of lesssignificance. E.g., the user device 8 can be a terminal device, such as Personal Computer, a smartphone,a tablet, notebook, or the like. The computing device 2 can for example be a server or a server farm or apart thereof. The processor 22 of the computing device 2 may comprise spatially distributed storage andprocessing means operatively connected with each other to implement the instructions stored in the memory 21. Referring to Figs.1A-B, a graphical user interface (“Qritos”) provided by the user device 2 allows a user toprovide an input data set. For example, after a new project is created by the computing device 2, the usermay add data, e.g., by uploading a file containing the input data set.According to the non-limiting example illustrated in Figs.1A-E, it is intended to obtain a control data set forcontrolling the production of a cake. The process would be same for any other product. The input data set comprises a first target property value related to a first product property of the product, and a second target property value related to a second product property of the product. The input data set further data related to a first input variable and a second input variable, wherein first dependencies between the first input variable and the first and second product properties are different from second dependencies between the second input variable and the first and second product properties. Referring to the example of the cake, cf. Fig.1C, the input variables are “wheat_flour”, “spelt_flour”, “sugar”, “chocolate”, “nuts” and “carrot”. For these input variables, the user enters data, namely the respectiverange of allowed values, cf., e.g., for “sugar”, the allowed values range from 0,15 to 0,35, e.g., indicating aweight share of sugar allowed in the cake. In the example, the user further defines three target properties,namely target values related to “calories”, “taste” and “browning”. The input variables “wheat_flour”, “spelt_flour”, “sugar”, “chocolate”, “nuts” and “carrot” have different effects on the target properties“calories”, “taste” and “browning”. In other words, the dependencies between the input variables“wheat_flour”, “spelt_flour”, “sugar”, “chocolate”, “nuts” and “carrot” on each of the target properties “calories”, “taste” and “browning” differ from each other. Referring to Fig.1D, the input data set in this example further comprises constraint data, the constraint data indicating one or more inadmissible values related to at least one of the first input variable and the second input variable. E.g., referring to the larger circle illustrated in Fig.1D, the user may indicate a mixture constraint related to the six ingredients, i.e., the input variables “wheat_flour”, “spelt_flour”, “sugar”, “chocolate”, “nuts” and “carrot”. Once the computing device 2 has received the input data set, the computing device 2 feeds the input data set to a model to obtain, from the model, the output data set. The output data set includes a plurality ofresults, wherein each results indicates result values of the product properties (“calories”, “taste” and“browning”) and associated values of the input variables (“wheat_flour”, “spelt_flour”, “sugar”, “chocolate”,“nuts” and “carrot”). For example, the output data set includes at least a portion of a Pareto front (also called Pareto frontier or Pareto curve) that has been obtained based on the input data set and the model. The Pareto front may include at least a portion of all Pareto efficient solutions. Referring to Fig.1E, the results are presented, atthe user device 8, in the form of value ranges, wherein the lower portion of Fig.1E illustrates the values ofthe input variables (there referred to as “Inputs”: “wheat_flour”, “spelt_flour”, “sugar”, “chocolate”, “nuts” and “carrot”), and the result values of the product properties (“calories”, “taste” and “browning”) are presented inthe upper portion of Fig.1E (there referred to as “Objectives”). Each result, i.e., combination of the threevalues of the target properties related to “calories”, “taste” and “browning” is associated with a corresponding set of values of the input variables “wheat_flour”, “spelt_flour”, “sugar”, “chocolate”, “nuts” and “carrot”. The user or the computing device 2 (or the computing device 2 itself) may select one of theresults considered most appropriate. After the selection, the corresponding data set of the selected resultmay be saved.Based on the selected result, the computing device 2 may derive the control data set for controllingproduction of the product (in this example: the cake).Fig.2 illustrates another method of providing a control data set for controlling the production of a product.There, the input data may be received (step 10) as exemplarily explained with respect to Fig.1A-D or inanother way. After the input data has been received, it is fed to the model, step 12, to obtain, step 13, theoutput data set. Before step 12, the model may be generated and / or selected from a plurality of available models based on the input data set. Different input data sets may require different models to obtain an appropriate output data set. For example, the model employed to obtain the output data set can be a multi- objective optimization model, e.g., a linear regression model, a partial least squares, PLS, model, a gaussian mixture model, a random forest model, a neural network model, e.g., a physics informed neuronal network model or the like. One of the problems associated with the use of a multi-objective optimization model is the plurality of results. Sometimes, several similar results are presented, wherein it may be difficult to decide, either for the computing device or for the user, which of the results shall be selected and subsequently form the basis for the generation of the control data set for controlling the production of the product. This problem is addressed by the embodiment illustrated in Fig.2. There, simultaneously with or after the obtaining the output data set (step 13), the computing device 2 determines, in step 14, for each of the each of the plurality of results, an associated quality score. In an embodiment, for each result, the quality score indicates a probability that the result is achieved within a certainty range, wherein said certainty range can be defined as part of the input data set. E.g., the input data set includes a certainty range, e.g., defined by a user, e.g., defining a range of 40% to 95%, meaningthat it is required that the result must be achieved with a certainty within the range of 40% to 95%. Therespective quality score in this case indicates, e.g., based on new observation / experiment carried out withthe step of determining the quality scores, the probability that the respective result is indeed with thespecified range. In another embodiment, each quality score indicates how reliable the associated result is,e.g., based on a prediction interval, an uncertainty may be determined. E.g., the larger the interval, the larger is the uncertainty associated with the result. For example, determining the associated quality scores includes an uncertainty estimation. E.g., a prediction interval estimation step is carried out after the model has provided the output data set, e.g., the Pareto points. E.g., based on each respective Pareto point, a prediction interval is calculated, so that the final pareto front has confidence bounds. These confidencebounds can be evaluated automatically by the computing device 2 and / or evaluated by a user. Based on the additional quality scores, a transparent selection criterion can be established. For example, it can be defined that only those results may be selected which have a respective associated quality score above a threshold level. For example, determining the quality scores is based on at least one of a first uncertainty value related to the input data set and a second uncertainty value related to the model. To this end, the input data set mayadditionally comprise experimental data about a measured or simulated relationship between (a) at leastone of the first input variable and the second input variable and (b) at least one of the first product property and the second product property. For example, based on the experimental data, the first uncertainty value related to the input data may be determined. E.g., referring to the example of Figs.1A-E, the input data may comprise one or more experimental results that have been achieved in precedent experiments, e.g., that a certain composition of the six ingredients has under certain conditions led to a certain set of values of the three properties (“calories”, “taste” and “browning”). In the examples of Figs.2 to 4, determining the associated quality scores can comprise applying a quantile regression method, e.g., on the output data set and the input data set. E.g., determining the associated quality scores excludes, in an embodiment, applying the method of least squares. In an embodiment, determining the associated quality scores is independent of the model used for obtaining the output data set. E.g., determining the associated quality scores is model agnostic. Forexample, the step of determining the associated quality scores may hence yield comparable quality scoreseven though different models may have been used to obtain the output data set.Fig.7 schematically illustrates a portion of an exemplary method of determining, step 14, quality scores inaccordance with one or more embodiments. In this embodiment, for determining the quality scores, said quantile regression method is applied.In step 140, the input data set is received (e.g., as part of step 10 described above). In this example, theinput data set comprises said data related to the first input variable and the second input variable, i.e., (i) data related to input variables. Furthermore, the input data set comprises (ii) said experimental data. In addition, the input data set comprises (iii) quantile range data indicative of a quantile range, e.g., said certainty range described above. For example, the quantile range is predefined. In another embodiment, the quantile range is selected, e.g., by the computing device 2 and / or by a user device 8, e.g., based on corresponding input from a user of the user device 8. For example, the quantile range data specifies one or more quantiles, such two quantiles, e.g., 5% and95%, or 10% and 80%. For example, the respective quantile defines a required probability for the results.E.g., the quantile range data, e.g., defined by a user, defines a range of 40% to 95%, meaning that it isrequired that each of the result must be achieved with a certainty within the range of 40% to 95%. Therespective quality score in this case indicates, e.g., based on new observation / experiment carried out withthe step of determining the quality scores, the probability that the respective result is indeed with the specified range. In step 142, a quantile regression model is generated. For example, the generated quantile regression model models the relationship between the input variables (e.g., independent predictor variables) and the specified quantiles (also called percentiles) of the target properties (e.g., dependent variables).Generating the quantile regression model in step 142 may occur in accordance with the following equation:^^(^^) = ^^(^) + ^^(^)^^^ + … + ^^(^)^^^ , wherein^^(^^) is the ^ -th quantile of the response variable / target property ^^^ is the quantile^^(^) are the regression parameters / the coefficient to be estimated and^^^, …, ^^^ are predictor / input variables for the ^-th observation.Further, generating the quantile regression model may include decreasing the mean absolute deviation,MAD, for Quantile Regression until the MAD is below a predefined value according to the followingequation: ^^^ =^∑^^ (^ − (^ (^^ ^^^^ ^ ^ ^ ) + ^^(^)^^^ + … + ^^(^)^^^)), wherein ^^is the function according to ^^(^) = ^ max(^, 0) + (1 − ^)max (−^, 0), wherein^ is the error of a single data point andmax() returns the largest value in the parentheses, e.g., u,0 or -u,0.Once the quantile regression model has been generated in step 142, the quantile range data and the data related to the input variables are fed into the quantile regression model, cf. step 144 in Fig.7. In an embodiment, also the results are fed into the quantile regression model. In step 146, the quality scores are obtained. As explained above, each result may be associated with at least one quality score. In a further embodiment, more than one method is applied to determine the quality scores. E.g., in addition to the quantile regression method, a gaussian process is applied to determine the quantile regression method. For example, the different quality scores obtained by applying the different methods are taken into account to determine a final quality score for each result, in accordance with an embodiment.Reverting to Fig.2, the model is applied to the input data set to obtain the output data set including theresults, as exemplarily described above, step 13. Further, for each of the results, the associated quality score is determined (step 14). The method of Fig.2 further comprises causing (step 15), by the computing device 2, rendering of a representation of the plurality of results and the associated quality scores. Such presentation may include a visual presentation, e.g., as exemplarily illustrated in Fig.1E. However, in addition to the results, theassociated quality scores are represented, e.g., based on error bars or confidence bounds. For example, inFig.3, the left graph shows an entire Pareto Front for two input variables y1 and y2 without confidencebounds, and the right graph illustrates the same Pareto Front with confidence bounds (dashed lines aboveand below the Pareto Front). The higher distance between the confidence bounds or, respectively, thelarger the prediction interval, the lower the quality of the respective Pareto Point. As explained above, the selection of the result occurs based on the quality scores. This selection may be carried out automatically by the computing device 2 or the by the user. In the latter case, the results and the associated quality scores are preferably graphically presented to the user, e.g., via the user device 8. Based on the graphic representation, a navigation service may be provided to the user allowing the user to scroll through the results and the associated quality scores. A further embodiment of the method 1 of providing a control data set for controlling the production of a product is illustrated in Fig.4. The method can be summarized as follows: Step 10 comprises receiving, by the computing device 2, the input data set, wherein the input data set can be configured as exemplarily described above. Step 12 comprises feeding, by the computing device 2, the input data set into a model to obtain 13, from the model, the output data set, the output data set including the plurality of results, wherein the results can be configured as exemplarily described above. Step 14 comprises determining, by the computing device 2, for each of the plurality of results, an associated quality score. This determination can be carried out as described above. For example, determining the associated quality scores may be based on the input data set. Further, determining the associated quality scores may include carrying out at least one of (i) a linear regression method, (ii) a quantile regression method and (iii) a prediction interval generating model. In an optional step 17, it is checked whether the quality scores are acceptable or not, e.g., by carrying out a threshold test. If it is determined that the quality scores are not acceptable, the method may comprise the step 11 of selecting 11, by the computing device 2, another model from a plurality of available models. Alternatively, a model considered appropriate may be generated. Steps 12 and 14 are then repeated until itis determined the quality scores are acceptable. In another embodiment, the model used to obtain theoutput data set (step 12) is not changed. For example, if the desired result cannot be achieved by the model with the specified certainty range, then this result might not be achievable in reality, assuming that the applied model accurately represents reality. On the other hand, if a new observation contradicts the model, then the model may be updated or changed to a better suiting one. Step 18 comprises selecting, by the computing device, one or more of the plurality of results based on the associated quality scores. As described above, this selection step may be carried out automatically by the computing device and / or in dependence of user input. Finally, step 19 comprises deriving 19, by the computing device 2, said control data set from the one or more selected result / s. The derived control data set is configured to control the production of the product. E.g., the derived control data set comprises process parameters related to the production process and / or indications of the amount or other characteristics of educts employed for producing the product.Fig.6 schematically and exemplarily illustrates a system 3 for producing a product P in accordance withone or more embodiments. Of course, the specifics of the system 3 depend on the product P to be produced and are herein not explained. In an embodiment, the system 3 is a chemical production plant.Fig.6 illustrates that the system 3 produces the product P based on a plurality of N educts ED_1 to ED_N.The production is controlled system parameters SP that, e.g., define processing of the educts ED_1 to ED_N based on one or more system processing units of the system 3. In accordance with the embodiment illustrated in Fig.7, the system 3, is furthermore controlled based on the control data set CDS that has been obtained in accordance with an embodiment described above. For example, the control data set CDSdefines at least one processing parameter for at least one of the system processing units of the system 3that is employed to process at least one of the N educts ED_1 to ED_N.

Claims

CLAIMS1. A method (1) of providing a control data set for monitoring and / or controlling a production and / or aprocessing of a product, the method (1) comprising:- receiving (10), by a computing device (2), an input data set, the input data set comprising:o a first target property value related to a first product property of the product, and asecond target property value related to a second product property of the product;o data associated with a first input variable and a second input variable, wherein one ormore first dependencies between the first input variable and the first and secondproduct properties are different from one or more second dependencies between thesecond input variable and the first and second product properties; and- feeding (12), by the computing device (2), the input data set into a model to obtain (13), fromthe model, an output data set, the output data set including a plurality of results, each resultindicating: oa first result value of the first product property, a second result value of the secondproduct property; and oa third value of the first input variable and a fourth value of the second input variable;and -obtaining, in particular determining (14), by the computing device (2), for each of the pluralityof results, an associated quality score; -selecting (18), by the computing device (2), one or more of the plurality of results based on theassociated quality scores; and -deriving (19), by the computing device (2), said control data set from the one or more selectedresult / s, optionally providing said control data set in particular for monitoring and / or controlling a production and / or a processing of a product.

2. The method (1) of claim 1, wherein the step of determining (14) the associated quality scorescomprises applying a quantile regression method.

3. The method (1) of claim 1 or 2, wherein determining (14) the quality scores occurs independently ofthe model used for obtaining the output data set, in particular independent of a model type associated with the model used for obtaining the output data set.

4. The method (1) of one of the preceding claims, wherein determining (14) the quality scores is basedon at least one of a first uncertainty value related to and / or associated with the input data set and asecond uncertainty value related to and / or associated with the model.

5. The method (1) of one of the preceding claims, wherein the quality score is indicative of a reliabilityof the respective result and / or wherein the quality score includes at least one prediction interval.

6. The method (1) of one of the preceding claims, wherein the input data set additionally comprisesconstraint data, the constraint data indicating one or more inadmissible values related to and / orassociated with at least one of the first input variable and the second input variable.

7. The method (1) of one of the preceding claims, wherein the input data set additionally comprisesexperimental data about a measured or simulated relationship between (a) at least one of the firstinput variable and the second input variable and (b) at least one of the first product property and thesecond product property.

8. The method (1) of one of the preceding claims, wherein the method further comprises:- selecting (11), by the computing device (2), the model from a plurality of available models.

9. The method (1) of claim 8, wherein said selecting (11) of the model is carried out based on thequality scores.

10. The method (1) of one of the preceding claims, wherein the model is hosted by the computing device(2).

11. The method (1) of one of the preceding claims, wherein the model is a multi-objective optimizationmodel.

12. The method (1) of one of the preceding claims, further comprising:- causing (15), by the computing device (2), rendering of a representation of the plurality ofresults and the associated quality scores; and, optionally, wherein: -the presentation includes at least one of a visual presentation, e.g., including error bars, andan acoustic presentation; and / or -wherein the presentation is provided by a user device (8).

13. A computing device (2), wherein the computing device (2) is configured to carry out the method (1)according to one of the preceding claims.

14. A method of producing a product, wherein production of the product is carried out based on a controldata set provided by a method (1) according to one of the preceding claims 1 to 12.

15. A system (3) for producing a product, wherein the system (4) is configured to carry out the method

Images