Method for designing hardware blocks and associated devices

Abstract

The present invention relates to a method for designing hardware blocks of a physical system comprising a prediction device predicting an output for given inputs, the method comprising a step of: - obtaining datasets corresponding to the outputs given by the device in the presence of the inputs of the dataset; - receiving the probability that a dataset is observed during a predefined use; - collecting the predicted outputs; - determining the distribution of the prediction accuracy of the predicted output for each dataset; - aggregating the determined distributions; - applying a risk metric to the time evolution of the aggregated prediction accuracy distribution in order to obtain a quantity representative of the performance range of the device; and - modifying the blocks according to the quantity obtained.

Description

[0001]TITLE: Method for Designing Hardware Blocks and Associated Devices The present invention relates to a method for designing the hardware blocks of a physical system adapted for a predefined use. The invention also relates to a computer program product and a readable information medium. The present invention falls within the field of developing critical embedded systems using prediction devices implementing an algorithm learned through machine learning. Machine learning is referred to by many different terms, such as the English term "machine learning," "automatic learning," "artificial learning," or "statistical learning." Machine learning consists of using data to learn a prediction algorithm. However,If the prediction algorithm thus learned performs well for the dataset used for its training, this does not demonstrate that the prediction algorithm is suitable for the intended use case. Various approaches exist for evaluating algorithmic performance, such as conducting robustness analysis experiments or using formal methods for certification within a predefined data range. However, these approaches only provide partial answers to the measurement problem and do not account for all use cases, making them imprecise. This leads, in particular, to embedded systems that are not suited to the desired performance in their intended use. Therefore, there is a need for a design process for the hardware components of a physical system adapted to a predefined use, enabling the creation of a more suitable physical system. To this end,The description describes a method for designing the hardware blocks of a physical system adapted for a predefined use, the use corresponding to a range of possible values ​​for several data points, the physical system including a prediction device, the prediction device being capable of predicting, for given inputs, the value of one or more outputs, the method comprising: - a phase of obtaining at least one quantity for a given configuration of the hardware blocks of the physical system, a configuration being defined by the value of the set of properties characterizing the blocks, the obtaining phase being implemented by computer and comprising the steps of: - obtaining datasets, each data point of a dataset corresponding to the output values ​​that the prediction device should give in the presence of the input values ​​of the dataset,- Receiving the probability for each dataset that a dataset will be observed during the predefined use for the given configuration, - collecting the outputs predicted by the prediction device for each input value of the datasets, - determining the distribution of the prediction accuracy of the predicted output for each dataset, to obtain determined distributions, - aggregating the determined distributions by using an aggregation function based on the received probabilities, to obtain an aggregated prediction accuracy distribution; the aggregation function is a weighted sum whose weights depend on the received probabilities, - determining a possible temporal variation of the weights with a Brownian motion simulation based on the weights used in the aggregation step.to obtain a time evolution of the aggregation function resulting in a time evolution of the aggregated prediction accuracy distribution, - application of at least one risk metric to the time evolution of the aggregated prediction accuracy distribution to obtain at least one quantity representative of the domain in which the prediction device has a predefined performance threshold, and - a phase of modifying the blocks of the physical system as a function of the quantity obtained. According to particular embodiments, the design process has one or more of the following characteristics, taken individually or in all technically possible combinations: - a boundary is defined for the domain,A quantity obtained in the application step is a representative measure of the temporal variation of the boundary size over time. - The domain is a set of datasets representative of the predefined use case; a quantity obtained in the application step is the time interval during which the performance of the prediction device on the set of datasets remains above the performance threshold. - A quantity obtained in the application step is, for a dataset, whether or not the dataset belongs to the domain in which the prediction device exhibits a predefined performance threshold. - The simulation involves an exponential temporal variation of the weight compared to the weight of the aggregation step. - The application step also includes an additional operation of statistical analysis of the risk metric result. - During the statistical analysis operation,An element is obtained from the expected value of the upper bound of the result, a predefined quantile of the result, and a statistical moment of the statistical distribution of the result. - The obtaining step is implemented by generating each dataset from a reference dataset according to a given probability law, or by generating initial data using a generative model and selecting the initial data according to a given probability law to form the dataset. - The obtaining step involves modifying the datasets by introducing imperfections in the physical system's environment or by introducing adverse perturbations aimed at manipulating the outputs of the prediction device. - The blocks of the physical system are chosen from a list consisting of a sensor, a memory, a processor, and a storage element.of a display unit and an input unit. - The properties characterizing the blocks are chosen from a list consisting of size, computing capacity, and sensitivity dynamics. - At least one of the inputs and / or outputs are physical quantity data corresponding to measurements from one or more sensors. - The process includes: - the iterative implementation of the acquisition and modification phases until a predefined criterion is met, to obtain a preferred configuration, and - the fabrication of the physical system according to the preferred configuration. The description also describes a computer program product comprising a readable information carrier, on which a computer program including program instructions is stored.The computer program is loadable onto a data processing unit and implements a method as previously described when the computer program is implemented on the data processing unit. The description also proposes a readable information carrier comprising program instructions forming a computer program, the computer program being loadable onto a data processing unit and implementing a method as previously described when the computer program is implemented on the data processing unit. In this description, the expression "suitable for" means interchangeably "adapted for," "tailored to," or "configured for." Features and advantages of the invention will become apparent from the following description, given solely by way of non-limiting example, and made with reference to the accompanying drawings.of which: - Figure 1 is a schematic representation of a system and a computer program product, and - Figure 2 is a flowchart of an example of the implementation of a process for designing the hardware blocks of a physical system adapted for a predefined use. A system 10 and a computer program product 12 are shown in Figure 1. The interaction between the system 10 and the computer program product 12 enables the implementation of a process for designing the hardware blocks of a physical system adapted for a predefined use. The design process is thus a computer-implemented process, or more precisely, the derivation phase of which is computer-implemented. The system 10 is a desktop computer. Alternatively, the system 10 is a rack-mounted computer, a laptop, a tablet, a personal digital assistant (PDA), or a smartphone. In specific embodiments,The computer is adapted to operate in real time and / or is in an embedded system, particularly in a vehicle such as an aircraft. In the case of Figure 1, the system 10 comprises a computing unit 14, a user interface 16, and a communication device 18. The computing unit 14 is an electronic circuit designed to manipulate and / or transform data represented by electronic or physical quantities in registers of the system 10 and / or memories into other similar data corresponding to physical data in register memories or other types of display devices, transmission devices, or storage devices. As specific examples, the computing unit 14 includes a single-core or multi-core processor (such as a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, and a digital signal processor (DSP)).a programmable logic circuit (such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic device (PLD), and programmable logic arrays (PLAs)), a state machine, a logic gate, and discrete hardware components. The computing unit 14 includes a data processing unit 20 adapted for processing data, including performing calculations, memories 22 adapted for storing data, and a reader 24 adapted for reading computer-readable media. The user interface 16 includes an input device 26 and an output device 28. The input device 26 is a device that allows the user of the system 10 to input information or commands onto the system 10. In Figure 1, the input device 26 is a keyboard. Alternatively, the input device 26 is a pointing device (such as a mouse, touchpad, or graphics tablet).a speech recognition device, an eye tracker, or a haptic (motion analysis) device. The output device 28 is a graphical user interface, i.e., a display unit designed to provide information to the user of the system 10. In Figure 1, the output device 28 is a display screen enabling a visual presentation of the output. In other embodiments, the output device is a printer, an augmented and / or virtual display unit, a loudspeaker or other sound-generating device to present the output audibly, a unit producing vibrations and / or odors, or a unit adapted to produce an electrical signal. In one specific embodiment, the input device 26 and the output device 28 are the same component forming human-machine interfaces.such as an interactive display. The communication device 18 enables unidirectional or bidirectional communication between the components of the system 10. For example, the communication device 18 is a bus communication system or an input / output interface. The presence of the communication device 18 allows, in certain embodiments, the components of the system 10 to be distant from each other. The computer program product 12 includes a computer-readable medium 32. The computer-readable medium 32 is a tangible device readable by the reader 24 of the computing unit 14. In particular, the computer-readable medium 32 is not a transient signal in itself, such as radio waves or other freely propagating electromagnetic waves, such as light pulses or electronic signals. Such a computer-readable storage medium 32 is, for example, an electronic storage device,a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. As a non-exhaustive list of more specific examples, computer-readable storage media 32 includes mechanically coded devices, such as punched cards or embossed structures in a groove, floppy disks, hard disk drives, read-only memory (ROM), random-access memory (RAM), read-only erasable memory (EROM), electrically erasable and readable memory (EEPROM), magneto-optical disks, static random-access memory (SRAM), compact discs (CD-ROMs), digital multipurpose discs (DVDs), USB flash drives, floppy disks, flash memory,a solid-state drive (SSD) or a PC card such as a PCMCIA memory card. A computer program is stored on the computer-readable storage medium 32. The computer program comprises one or more sequences of stored program instructions. Such program instructions, when executed by the data processing unit 20, cause the execution of steps in the design process. For example, the form of program instructions is a source code form, a computer-executable form, or any intermediate form between source code and a computer-executable form, such as the form resulting from the conversion of source code via an interpreter, assembler, compiler, linker, or locator. Alternatively, program instructions are microcode, firmware instructions, state definition data,Integrated circuit configuration data (e.g., VHDL) or object code. Program instructions are written in any combination of one or more languages, for example, an object-oriented programming language (FORTRAN, C++, Java, HTML), a procedural programming language (e.g., C). Alternatively, program instructions are downloaded from an external source via a network, as is often the case with applications. In this case, the computer program product includes a computer-readable data carrier on which the program instructions are stored or a data carrier signal on which the program instructions are encoded. In each case,The computer program product 12 includes instructions that can be loaded into the data processing unit 20 and adapted to trigger the execution of the design process when executed by the data processing unit 20. Depending on the embodiment, execution is carried out wholly or partially either on the system 10, i.e., a single computer, or in a distributed system across multiple computers (notably via the use of cloud computing). The operation of the system 10 is now described with reference to Figure 2, which is a flowchart illustrating an example of an implementation of the design process. The design process aims to design hardware blocks of a physical system suitable for a predefined use. In one example, the hardware block is a sensor. In another example, the hardware block is a memory. Alternatively,The hardware block is a processor. In another example, the hardware block is a storage element. Alternatively, the hardware block is a display unit. In yet another example, the hardware block is an input unit. More generally, a hardware block is any combination of the elements mentioned above that allows for the construction of a complex physical system. Each block has specific properties, and these properties characterize the block. Knowledge of these properties allows for the construction of each hardware block. For example, the properties characterizing the blocks are chosen from a list consisting of size, computing capacity, and dynamic range. The physical system is, for example, a computer physically connected to the data processing system, or the computer on which the data processing is performed. Alternatively,The physical system is a computing server connected via a network to the data processing system. In yet another example, the physical system is a server containing a virtual machine. Alternatively, the physical system is a portable device such as a smartphone or tablet. This physical system is designed for a predefined use. The use corresponds to a range of possible values ​​for various data points. This use may involve monitoring system behavior, controlling network security by monitoring associated flows, or controlling the selection of modules to be implemented based on the context to ensure processing. The physical system includes a predictive mechanism.The prediction device is designed to predict the value of one or more outputs for given inputs. The prediction device is a physical circuit implementing a prediction algorithm capable of predicting the value of one or more outputs for given inputs. The algorithm was learned using a machine learning technique and a training dataset. More precisely, in the example that will be described, the algorithm is a supervised statistical learning algorithm. In the following, such an algorithm is denoted by a function, ^^:^^→^^where the set X denotes the set of inputs of the algorithm and Y the set of outputs of the algorithm. The prediction algorithm is, for example, a support vector machine (SVM), a neural network (NL), or a random forest. More generally, any type of supervised prediction algorithm is suitable for this context. As a point of note, when the prediction algorithm is a neural network, the physical circuit is called a neuromorphic circuit. Such a prediction algorithm can be used in very diverse contexts such as image classification, three-dimensional pattern recognition, or decision support for autonomous drone piloting.Preferably, the prediction algorithm takes as input and / or outputs physical quantities corresponding to measurements from one or more sensors. The prediction algorithm can, for example, be based on autoregressive techniques such as those in the ARIMA family, on regression such as linear regression, on machine learning such as an LSTM neural network, or on a rule system such as a decision tree. The process comprises a generation phase P1, a modification phase P2, and a manufacturing phase P3. During the generation phase P1, the system obtains at least one quantity for a given configuration of the physical system's hardware blocks. A configuration is defined by the value of the set of properties characterizing the blocks.The configuration thus corresponds to a specific physical system, with the blocks respectively representing the value(s) of the configuration's properties. The magnitude is a measure of the physical system's performance relative to the use cases in which the physical system exhibits the expected performance. Typically, in the design of a physical system, it is expected that the physical system will behave well, for example, correctly identify a target, in 90% of the cases that may occur in practice. As we will see later in the description, the magnitude here represents the domain in which the prediction device exhibits a predefined performance threshold. The acquisition phase P1 comprises an acquisition step E40, a reception step E42, a collection step E44, a first determination step E46, an aggregation step E48, a second determination step E50, and an application step E54.During the E40 acquisition step, the system 10 obtains datasets. Each data point in a dataset corresponds to the output values ​​that the prediction device should produce given the input values ​​of the dataset. In this description, preferably, at least one of the inputs and / or outputs are physical quantity data corresponding to measurements from one or more sensors. In another example, the E40 acquisition step is implemented, for each dataset, by generating initial data using a generative model and selecting the initial data according to a given probability distribution to form the dataset. A generative model is a machine learning algorithm that seeks to describe the data, subsequently generating new samples according to the description (i.e., probability distribution) determined during the learning phase.A classic example is a generative adversarial network (GAN), which allows the synthesis of highly realistic (fictional) images from real images. Another example is a variational autoencoder (VAE). In other words, compared to the previous implementation, instead of using reference datasets, it uses data obtained with a generative model. Alternatively, or in addition, the E40 acquisition step involves modifying the datasets (generated or reference datasets) by introducing imperfections in the environment of the system that the prediction device models.For example, if the image to be recognized is a scanned image of a handwritten note, the datasets can be modified to account for imperfections in the scanner used. Other examples consider geometric transformations, noise, or external disturbances for modification. Regarding external disturbances, it should be noted that introducing adverse disturbances designed to manipulate the outputs of the prediction device significantly increases the robustness of the evaluation. Thus, in all cases, the E40 acquisition step results from the implementation of a generation of input / output pairs that yields various realizations of random variables (x,y) under the measure of... At the end of the acquisition step E40, a finite set of input / output pairs is obtained. During the reception step E42, the system receives the probability for each dataset that a dataset will be observed during the predefined use case for the given configuration. For example, if the datasets correspond to black and white images, while in the use case the images are in color, the case where the color images are comparable to the case of black and white images has a certain probability. Similarly, the distribution of digits in use is not equally distributed, so the probability of having some digits is higher than the probability of having other digits. During the collection step E44, the outputs predicted by the prediction device for each input value of the data from the datasets are collected.Put another way, the prediction device is applied to the input values ​​and the result is collected by system 10. For each input, the value predicted by the prediction device and the value that the prediction device should have predicted (true value) are thus known. During the first determination step E46, system 10 determines the distribution of the prediction accuracy of the predicted output for each dataset, to obtain determinate distributions. The prediction accuracy is obtained by applying an evaluation metric to a prediction error. Such an evaluation metric will be denoted φ in the following. A prediction error corresponds to the following quantity: Where: •x and y are realizations of the random variables x and y with joint distributionℙ ^^,^^ And • + is a precision metric, also called a loss function. According to the example described, the prediction precision is calculated using a metric with respect to the absolute prediction error. However, any method of calculating the prediction precision is conceivable at this stage. Thus, in one example, the prediction precision is calculated using a cross-entropy function. In another example, the prediction precision is evaluated using a quantile metric. More generally, the evaluation metric is constructed by applying a metric φ to the empirical distribution of the random variable μ(μ(μ),μ). In other words, the evaluation metric is an empirical moment of the distribution of the prediction precision. Alternatively, the prediction precision is calculated using a metric with a reference prediction device denoted g.According to a particular example, the evaluation metric is a function φ of the distribution of relative accuracy between the prediction device represented by the function f to be evaluated and the reference prediction device g. As an illustration, the metric is the average evaluation of the l-relative differences, which is mathematically written as follows: It is also possible to refine the previous metrics by conditioning them on the accuracy of the reference prediction device g. The conditional mean of the relative l-differences is a particular example of such a metric, which can be written mathematically as follows: The evaluation metric is calculated independently for each dataset. It should be noted that the evaluation metric can be viewed as a risk metric. At the end of the first determination step E46, the following is obtained ^^ ^^distributions of the prediction error of the predicted output. Each distribution is thus a specific distribution unique to a respective dataset sampled according to a distribution with i an integer between 1 and , with = × , where N represents the number of datasets used to obtain a realization of the prediction error calculation. Each of these determined distributions is denoted with i an integer between 1 and ^^ ^^ During the E48 aggregation step, the system 10 aggregates the determined distributions to obtain an aggregated prediction accuracy distribution. Thus, during the E48 aggregation step, the determined distributions ^^ ^ ℙ ^ , ^ ^ ^ ^ ^ ,^ ^^ ^ ^ are aggregated to obtain an aggregate prediction error distribution or aggregate prediction distribution denoted ^^ ^ℙ ^ , ^ ^ ^ ^ ,^^ ^ ^ To do this, an aggregation function is used, employing the probabilities received. For example, the aggregation function is a weighted sum whose weights ^^ ^^ depend on the probabilities received. These weights are sometimes called priors. ^^ ^^ corresponding to each of the probability measures (ℙ^^ ^ ,^ ^^) ^^^^ ^^ = 1. Mathematically, this amounts to constructing a distribution At the end of the aggregation step E48, an aggregated distribution of the prediction accuracy is obtained. During the second determination step E50, the system 10 determines a possible variation of the weights using a Brownian motion simulation based on the weights used in the aggregation step E48. The formalism associated with the implementation of the second determination step E50 is now introduced. This step aims to simulate the process. In the case of a linear aggregation function, such that ℙ = ∑ℙ = 1 / ℙ, and the aggregation constraint is then written as: The following modeling is then used: Or : is a standard Brownian motion• ^^ < ^^^^^ ^,^^^^^^ >= ^^^^,^^^^^^ for ^^ ≠ ^^ ,• h^^ is a positive deterministic function of time, and• ^^^^ = ∑^^ ^^2^^ = 1h^^(^^)^^0 ^^ ^^exp( ‒ 2 ^^ + ^^^^^^^^^^) is a (stochastic) denormalization factor. Thus, according to the example described, the simulation involves an exponential temporal variation of the weight compared to the weight obtained after implementation of the E48 aggregation step. This modeling is accompanied by calibration and discretization, which are now described. For calibration, it can be observed that the aggregation constraint is satisfied with the formalism previously introduced. This formalism also allows us to formulate constraints on the average value of the different components, using, for example, piecewise constant functions: ℎ^^(^^) = ℎ^ ^^^1{^^^^ ≤ ^^ < ^^^^ + 1}. Thus, it can, for example, be postulated that the diffusion oscillates around the initial values. ^^1 0 ,...,^^^ 0 ^(considered as average values) by jointly solving the following equations, for ^^ = This corresponds mathematically to the following expression for the expectation ^^ : where the expectation can be calculated via a standard Monte Carlo method by simulating the variables (^^ ^ ^^ ^ ^^ ) ^^ = 1 like Gaussians centered with variance ^^ and correlated according to the constraint ^^ < ^^^^^ ^,^^^^^ ^ >= ^^^^,^^^^^^ . Regarding discretization, it is necessary to simulate the trajectories of the process ( ^^ ^^ ) ^^ ≥ 0 for a noted time horizon ^^ ^^ This time horizon ^^ ^^is different from the horizon used for calculating the risk indicator. To do this, two approaches are described here. 5 The first approach is an ad-hoc simulation using the very specific form of the integrated process. The discretization grid is a discretization grid with constant step size Δ = 0, 1, 1 − 1. As , the trajectories of components can be jointly simulated via the generation of centered reduced Gaussians Gi,k , with corr(Gi,k,Gi,k) = ^^^^,^^ . Thus: with the initial condition ^^^^ ^^0 = ^^0 ^^. The second approach uses the direct simulation of the differential equation 15stochastic analysis of the process under consideration using Euler-type or other discretization schemes. To clarify this approach, a stochastic differential equation is derived for each component based on d'Itô's formula. This yields: ℎ ... ^ ( ‒ 2 ^^ + ^^^^^^^^^^)^^( ^^ ) + ^^0 ^^ ^^ ^^ ^^ exp( 20. Noting vient Furthermore, the following relationship is verified: 25 The dynamics of ^^ ^^ s^is obtained with the following relation: Therefore, it follows that: This implies that: Substituting the previous equations leads to the following mathematical relationship: Combining all these elements allows us to establish the following stochastic differential equation: I l vient ainsi : ^ ^^^^^^ ^ D ^où il résulte que : ^ ^^^^^^ ^ ^ ^^^ This corresponds to the following expression: By noting ^^^^ = (^^1^^,...,^^^^^^) , the previous expression becomes: Where: o^^^^ is the contribution of the variation of ^^^^^^ which depends on the variation over time, and o is the contribution of the variation of ^^^^^^ which does not depend on it. The formalism introduced thus makes it possible to obtain a time evolution of the aggregation function. At the end of the second determination step E50, the system 10 thus obtains a time evolution of the aggregation function resulting in a time evolution of the aggregated prediction accuracy distribution. During the application step E54, the system 10 applies at least one risk metric to the time evolution of the aggregated prediction accuracy distribution to obtain at least one quantity representative of the domain in which the prediction device has a predefined performance threshold. Before explaining specific implementation methods for such an application step E54, the mathematical concepts involved in this implementation can be explained here.The stratification of the distribution ℙ also extends to the error distributions ^^^^,ℙ , via the existence of a function ^^ defined by:. The function ^^ thus allows the calculation of the risk indicator via the following formula: Given a trajectory for the completion of the process (^^ ^^ ) ^^ ≥ 0 , it is then possible to obtain a trajectory ( via the previous formula ℛ^^ ^^ , 1ℙ (^^),...,ℛ^^ ^^ , ^ ℙ ^ (^^) ) corresponding for the values ​​of the risk indicator. The risk indicator ℛ ^^ ^ , ^ ℙ (^^) thus corresponds to the temporal evolution of the aggregate prediction accuracy distribution for the prediction device ^^ The risk indicator ℛ^^ ^ , ^ ℙ (^^)is a random variable and thus allows the calculation of various metrics, known as risk metrics, and the implementation of a statistical analysis of the risk metric's outcome. For example, during this operation, the system determines the expected value of the upper bound of the outcome. In another example, the system determines a predefined quantile of the outcome. In yet another example, the system determines a statistical moment of the statistical distribution of the outcome. Examples include the variance, the quantiles of the term distribution, or the mean of the extreme trajectories over a given time interval. More generally, during the statistical analysis, the system determines a statistical value related to the statistical distribution of the outcome. Some formulas are developed below to illustrate these examples.For a risk horizon ^^ , uncertainty bounds for ℛ^^0,ℙ (^^) can be defined, by considering for example. An upper bound can also be established at a given confidence level using the quantile term. ^^ (ℛ ^^ ^ , ^ ℙ (^^)) . The exploitation of the trajectories obtained by simulation also allows us to refine the estimation of the uncertainty by considering, for example, the average of the extreme values ​​of the indicator given by ^^( sup^^ ∈ [0,^^] ) . For a risk tolerance characterized by the level ^^0 and the confidence level^^, the notion of critical usage time ^^^ ^ ^^ 0(^^) of the prediction device ^^ can also be introduced as the time ^^ after which it can be guaranteed (for a given confidence level ^^) that the risk ℛ^^ ^,^ℙ (^^) is less than level ^^0. This translates mathematically to: ^^0) ≥ ^^} Where: oℚ is the probability distribution associated with the specification of the diffusion of the process Exploiting all trajectories also allows us to consider an alternative indicator for critical usage time, by considering: ^ ^^ ^ ^^ ^ 0(^^) = ^^[inf{^^ ≥ 0, ℛ ^ ^, ^ℙ (^^) > ^^0}]Indicators ^^^ ^ ^^ 0(^^) and ^^^ ^ ^^ 0(^^) are calculated in practice using a Monte Carlo method based on the trajectories of simulated risk indicators derived from the diffusion of the process as explained previously. As a point of note, the modeling and associated stochastic differential equation introduced allow for the simulation of process implementation trajectories. It has also been illustrated how to obtain corresponding trajectories for the realization of the risk indicator, considered as a stochastic process. ( ℛ^^ ^ , ^ ℙ (^^) )^^ ≥ 0. In terms of probabilistic formalism, a trajectory is defined for ^^ ∈ ^^(set of^events) as the function ^^→ℛ ^^ ^ , ^ ℙ (^^)(^^) , which allows us to identify the values ​​leading to the event {ℛ^^ ^, ^ℙ (^^) > ^^0} .In other words, this is how we obtain the realizations ^^^^ of a random variable ^^such that ^^^ i^ = ^^^^k(^^^^) for which standard analysis techniques such as principal component analysis can be implemented to highlight certain patterns and dependencies between the components of the vector ^^ resulting in risk indicators exceeding the level ^^A predefined quantity is defined. As a specific example, a boundary is defined for the domain, and a quantity obtained is a representative measure of the temporal variation of the boundary size over time. This corresponds to obtaining a quantity representative of the accuracy of the physical system. In another example, the domain is a set of datasets representative of the predefined use case, and a quantity obtained is the time interval during which the performance of the prediction device on the set of datasets remains above the performance threshold. In yet another example, a quantity obtained is, for a given dataset, whether or not the dataset belongs to the domain in which the prediction device exhibits a predefined performance threshold. These two examples correspond to obtaining a quantity representative of the performance of the physical system.It is possible that the system 10 obtains one or more of the aforementioned quantities during the application step E54. At the end of the acquisition phase P1, a quantity representative of the domain in which the prediction device exhibits a predefined performance threshold for a given configuration is thus obtained. This means that, for each of the properties of the physical system's blocks, the system 10 associates a quantity that characterizes whether the predefined performance threshold is satisfactorily met by the physical system in the predefined use case. During the modification phase P2, the physical system's blocks are modified according to the quantity obtained during the acquisition phase P1. For example, the computing capacity is increased when it is determined that the representative quantity does not correspond to satisfactory performance.The idea here is to modify the property values ​​(change the configuration) until the physical system can meet the predefined performance threshold. The process involves an iterative implementation of the acquisition phase (P1) and the modification phase (P2) until a predefined criterion is met, resulting in a preferred configuration (for example, the one with the highest value). The fabrication phase (P3) is the phase in which the physical system is manufactured according to the preferred configuration. In other words, the physical system is manufactured using hardware blocks with the property values ​​of the preferred configuration. The fabrication phase (P3) is carried out by obtaining the hardware blocks and assembling them to obtain the physical system. For example, memory and processors with the determined properties are retrieved, and electrical connections are made between these elements, resulting in a computer.This method is a tool-based approach for measuring and managing the long-term risk associated with the use of artificial intelligence algorithms in the design of a physical system. At the end of the validation phase, the method provides a realistic measurement of the physical system's performance across all data that can be submitted to it, unlike a certification process that would only validate the system's performance under specific usage conditions. The method achieves good accuracy for all possible practical scenarios. This method has the advantage of being generic, meaning it can be applied to any type of predictive device in a supervised learning context, any type of input, and any intended use case.The process was implemented by the applicant using several modules: a dataset retrieval module, a reception module, a predicted value collection module, a first determination module, an aggregation module, a second determination module, and an application module. Each of these modules was successfully implemented in the Python programming language. However, any type of object-oriented language, particularly one with polymorphism, would also provide good operational efficiency. This modular implementation makes the process easily adaptable to all types of algorithms, since each module is relatively independent. Furthermore, the process is easily parallelized, which helps to limit the computational load, notably through the use of a distributed computing infrastructure.It should be noted that parallelization is implemented at the level of the various parameters used to define the scenarios, as well as at the level of calculating the different associated distributions, with each realization of a given scenario being able to be processed individually. Other implementations are possible. In one implementation, the process also includes displaying all of this information on the output device 28, which then serves as a graphical user interface. Such a display would replace or complement the generated report by allowing the user to view the various results and performance indicators and to conduct detailed analyses by enabling navigation through the different results files and modularity in the selection of risk metrics.In particular, the user can, if desired, convert a given level of algorithmic risk tolerance into validity ranges for the algorithms under consideration by analyzing the contributions of different scenarios. Alternatively, or in addition, the graphical interface allows the creation of a configuration file containing all the information necessary for implementing the process, including information on the acquisition and reception steps. For this purpose, output device 28 allows the user to enter the associated data or select it using drop-down menus. As a specific example, the configuration file includes the parameters necessary for defining the risk metrics to be calculated, the algorithms to be evaluated, the accuracy metrics to be considered, the different datasets to be considered, the simulation methods for the datasets, the reference algorithms, and the priors.Finally, it should be clearly understood that the order of steps in the design process just described may differ, and in particular that some steps may be carried out simultaneously. More generally, any technically feasible combination of the preceding embodiments allows for a design process for the hardware blocks of a physical system adapted for a predefined use.

Claims

CLAIMS 1. A method for designing the hardware blocks of a physical system adapted for a predefined use, the use corresponding to a range of possible values ​​for several data, the physical system comprising a prediction device, the prediction device being capable of predicting, for given inputs, the value of one or more outputs, the method comprising: - a phase of obtaining (P1) at least one quantity for a given configuration of the hardware blocks of the physical system, a configuration being defined by the value of the set of properties characterizing the blocks, the phase of obtaining (P1) being implemented by computer and comprising the steps of: - obtaining (E40) datasets, each data point of a dataset corresponding to the output values ​​that the prediction device should give in the presence of the input values ​​of the dataset,- Receiving (E42) the probability for each dataset that a dataset will be observed during the predefined use for the given configuration, - Collecting (E44) the outputs predicted by the prediction device for each input value of the data from the datasets, - Determining (E46) the distribution of the prediction accuracy of the predicted output for each dataset, to obtain determined distributions, - Aggregating (E48) the determined distributions by using an aggregation function based on the received probabilities, to obtain an aggregated prediction accuracy distribution; the aggregation function is a weighted sum whose weights depend on the received probabilities, - Determining (E50) a possible temporal variation of the weights with a Brownian motion simulation based on the weights used in the aggregation step.to obtain a time evolution of the aggregation function resulting in a time evolution of the aggregated prediction accuracy distribution, - application (E54) of at least one risk metric to the time evolution of the aggregated prediction accuracy distribution to obtain at least one quantity representative of the domain in which the prediction device has a predefined performance threshold, and - a modification phase (P2) of the blocks of the physical system as a function of the quantity obtained.

2. A method according to claim 1, wherein a boundary is defined for the domain, a quantity obtained at the application step (E54) being a representative measure of the temporal variation of the size of the boundary over time.

3. A method according to claim 1 or 2, wherein the domain is a set of datasets representative of the predefined use, a quantity obtained at the application step (E54) being the time interval during which the performance of the prediction device on the set of datasets remains above the performance threshold.

4. A method according to any one of claims 1 to 3, wherein a quantity obtained at the application step (E54) is, for a dataset, whether or not the dataset belongs to the domain in which the prediction device exhibits a predefined performance threshold. 5.

6. A method according to any one of claims 1 to 4, wherein the simulation involves an exponential time variation of the weight with respect to the weight of the aggregation step.

7. A method according to any one of claims 1 to 5, wherein the application step (E54) also comprises an additional operation of statistical analysis of the result of the risk metric.

8. A method according to claim 6, wherein, during the statistical analysis operation, an element is obtained from among the expected value of the upper bound of the result, a predefined quantile of the result, and a statistical moment of the statistical distribution of the result.A method according to any one of claims 1 to 7, wherein the data acquisition step (E40) is implemented by generating each dataset from a reference dataset according to a given probability distribution, or by generating initial data using a generative model and selecting the initial data according to a given probability distribution to form the dataset. A method according to claim 8, wherein the data acquisition step (E40) comprises modifying the datasets by introducing environmental imperfections. of the physical system or by introducing adverse disturbances intended to manipulate the outputs of the prediction device.

10. A method according to any one of claims 1 to 9, wherein the blocks of the physical system are selected from the list consisting of a sensor, a memory, a processor, a storage element, a display unit, and an input unit.

11. A method according to any one of claims 1 to 10, wherein the properties characterizing the blocks are selected from the list consisting of size, computing capacity, and sensitivity dynamics.

12. A method according to any one of claims 1 to 11, wherein at least one of the inputs and / or outputs are physical quantity data corresponding to measurements from one or more sensors. 13.A method according to any one of claims 1 to 12, wherein the method comprises: - the iterative implementation of the obtaining and modification phases until a predefined criterion is met, to obtain a preferred configuration, and - the fabrication of the physical system according to the preferred configuration.

14. A computer program comprising a readable information carrier, on which is stored a computer program comprising program instructions, the computer program being loadable onto a data processing unit and implementing a method according to any one of claims 1 to 13 when the computer program is implemented on the data processing unit. 15.Readable information carrier comprising program instructions forming a computer program, the computer program being loadable onto a data processing unit and implementing a method according to any one of claims 1 to 13 when the computer program is implemented on the data processing unit.

Images