Verification of a cyber-physical system (CPS)

Abstract

Method for verification of a cyber-physical system (15) being controlled by a controller (2), the method comprising: - operating the system (15) starting from multiple different initial states; - detecting during operation a parameter value of at least one parameter describing the state of the system; - recording the parameter values at corresponding relative moments over time as a state time series (7) for each of the initial states; - determining at least one cross-sectional comprising the parameter values of the same relative moment in all of the state time series; - determining at least one convex body (8) based on a pre-defined proportion of the at least one cross-sectional; and - determining the reliability of the system (15) based on the at least one convex body (8).

Description

[0001]Verification of a cyber-physical system (CPS)The invention concerns a method for verification of a cyber-physical system (CPS) being controlled by a controller. Inparticular, the method can be used to verify learning-basedcontrollers of cyber-physical systems, such as those employed inNeural-Network-Controlled Systems (NN-controlled CPS).Specifically, the invention proposes a method for statisticalverification of a cyber-physical system based on sampled datafrom actual observations of the system.The interest in verifying systems, in particular those operatedby learning-based controllers, is not new and only increaseswith the prevalence of such (in some way black-box) controllers.Specifically, when application of such systems in applicationsthat depend on safe and correct operation, a systematic andquantitative assessment of robustness, reliability, and safetyis desirable or even indispensable. Robustness tackles thequestion whether small perturbations affect the system; anassessment would ideally provide guarantees for the effect ofsmall changes in the input, e.g. from adversarial attacks, noiseetc., on output. Safety tackles the question whether the systemcan reach one or more unsafe states; an assessment would ideallyprovide the likelihood of such an event. The actual significanceof what is a small change or what is an unsafe state is asdiverse as the class of cyber-physical systems.Previous works aiming at providing such assessments haveimportant limitations that significantly impede broaderapplication in practice:GRUENBACHER S. et al. ("GoTube: Scalable Stochastic Verificationof Continuous-Depth Models" [online], arXiv:2107.08467, December2nd, 2021, XP091111344) propose a stochastic verificationalgorithm that relies on continuous-depth models to analyticallyderive a bounding tube (or “reach tube”) representing theboundaries of the parameter space reachable by the controllerwith a pre-defined confidence. The nature of the algorithmlimits the application to fully known, analytically definedcontrollers of a certain class.ZAREI M. et al. ("Statistical verification of learning-basedcyber-physical systems" In: Proceedings of the 23rdInternational Conference on Hybrid Systems: Computation andControl (HSCC ’20) [online], New York, NY, USA: Association forComputing Machinery, April 22nd, 2020, pages 1–7. ISBN 978-1-4503-7018-9. XP058715710.) propose the use of Statistical ModelChecking (SMC) for verifying complex NN-controlled CPS. At itscore the method relies on a simulation of the controller. Itinherently requires iterative sampling of the controllerstarting from a modelled distribution of initial states. Hence,this method also requires complete control over and access tothe controller in order to simulate its behavior to the requiredextent. In addition, the proposed SMC gives only a qualitativeresult with which guarantee the trajectories are going to beinside undesired states and does not provide a quantitativeassessment of how the system / the trajectories are going tobehave.A disadvantage of these methods is the reliance on an accuratesystem model.It is an object of the present invention to overcome or at leastalleviate some of the shortcomings of the prior art and providean assessment better suitable for use with actual measurementsand observations.The present invention proposes a method, as mentioned in theoutset, for verification of a cyber-physical system beingcontrolled by a controller, the method comprising:- operating the system (i.e., the cyber-physical system)starting from multiple different initial states;- detecting during operation a parameter value of at leastone parameter describing the state of the system;- recording the parameter values at corresponding relativemoments over time as a state time series for each of the initialstates; -determining at least one cross-sectional comprising theparameter values of the same relative moment in all of the statetime series;- determining at least one convex body based on a pre-defined proportion of the at least one cross-sectional; and- determining the reliability of the system based on the atleast one convex body.The controller of the system may in particular be a learning-based controller, e.g. a Neural-Network based controller. Theparameter(s) of which the values are detected and recorded (orat least some of those parameters) may optionally be used by thecontroller as (an) input parameter(s).Operating the system includes running the controller toinfluence the state of the system. In general, the system may bepart of a greater product or system, wherein the parts operatedfor the present method are at least those required for thefunctioning of the controller. Those parts typically include atleast the signals providing inputs required by the controllerand the signals providing the controller outputs. The number nof different initial states is two or more. The multipledifferent initial states may be obtained by uniform samplingwithin known limits of the system. Alternatively, the multipledifferent initial states may be obtained by „randomly“ startingthe system as it would be done in reality, i.e., without tightlycontrolling the initial state. This approach yields a stochasticdistribution of the initial states without intentional / directedsampling. Preferably, the multiple different initial statesshould statistically represent a real distribution of states ofthe system. For that purpose, data on visited states can begathered from the live system. The initial states can then besampled from the gathered data. To ensure a correctrepresentation, the sampled data can be tested against thedistribution of the gathered real data, for example using aKolmogorov–Smirnov test. Once the initial states are provided,the system started from each of them and operated for examplefor a particular time span or to a particular precondition ismet.During operation of the system, a parameter value of at leastone parameter describing the state of the system is detected.The number d of parameters is one or more. The parameter may beone that is relevant for the safety, reliability and / orrobustness of the system. The parameter may be directlymeasurable by means of a detector or sensor. The parameter mayalso comprise an indirect measurement, for example when theparameter values are composed of or derived from one or moredirect measurements and optionally further input parameters. Thetype and number of parameters depends on the type andfunctioning of system as well as on the desired assessment (forexample, the definition of any undesired states and theparameters used for that definition).The detected parameter values are then recorded at correspondingrelative moments over time as a state time series for each ofthe initial states. For example, for each execution of thesystem starting from one of the multiple different initialstates, the parameter values of the same parameters are detectedand recorded at similar durations after the start of theoperation. these moments do not need to be regular intervals.There are no restrictions on the relative moments within thesame state time series. The multiple state time seriescorresponding to the multiple different initial states need toshare at least one common relative moment at which the at leastone parameter value has been detected and recorded during eachof the separate executions corresponding to the initial states.Once the state time series have been recorded (at least twostate time series and more depending on the desired statisticalproperties of the assessment), at least one cross-sectional isdetermined that comprises the parameter values of the samerelative moment in all of the state time series. The cross-sectional (or each cross-sectional in case of more than one) isassociated with one relative moment and comprises the (d)parameter values of each of the at least one parameters recordedat that relative moment during the operation of the systemstarted from each of the (n) multiple initial states (hence, becross-sectional comprises at least d times n parameter values).At least one convex body is determined based on a pre-definedproportion of the at least one cross-sectional. The pre-definedproportion may be the entirety (or 100%). Alternatively, asmaller proportion may be used for determining the convex body,for example to exclude outlier parameter values based on aparticular pre-defined percentile or to exclude unsafe parametervalues of a pre-defined percentage of the cross-sectional.Outliers may be detected based on individual parameters, forexample wherein a recorded state is rejected when any of itsparameter values is an outlier within the distribution of theparticular parameter in the cross-sectional; alternatively,outliers may be detected based on a combined measure or metric,such as the ones discussed further below. Optionally, the atleast one convex body may be determined as enveloping the pre-defined proportion of the at least one cross-sectional. Theconvex body can be any compact convex set, whether symmetric(such as a hypersphere, hyper-ellipsoid, or hypercube) orasymmetric. The goal is that the convex body correctlyrepresents the parameter space reachable by the system from anyinitial state possible in reality. A correct representation doesgenerally not remove, but has a limited residual risk that thesystem behaves outside the convex body – and may reach (alsounsafe) states separate from the convex body. With our method,an upper bound of the residual risk is being estimated. Thesignificance indicates the probability of the residual riskbeing higher than estimated. Both, the significance and theresidual risk characterize the reliability determined using thepresent method, i.e. the assessment whether the convex body isseparate (or non-intersecting) from a pre-defined unsafe state.The usefulness of finding reliability is limited if thesignificance and / or the residual risk are / is high. Whenever asystem is called “reliable” in this disclosure, this means moreprecisely “reliable with a residual risk”, said residual riskbeing limited and having a known upper limit according to thepresent method. Specific embodiments of the present methodprovide an assessment of all of these indicators.Finally, the reliability of the system is determined based onthe at least one convex body. In other words, the outcome of theassessment of the reliability (including safety and robustness)of the system as a function of the at least one convex body isdetermined as described in the previous steps. The position,shape, and size of the convex body in the parameter space aredecisive for the assessment of the reliability.As becomes clear from the foregoing, in contrast to prior artmethods, the present method is generally applicable and providesa technical framework for the verification of one or more ofsafety, reliability, and robustness based on the observedbehavior of the cyber-physical system, without the need forcomplete information on the controller, and without arestriction to a narrow class of NN-based controllers. Theproperties of the controller and hence the control system can bederived from the actual behavior in a real, physicalenvironment. The present method applies generally to continuous-time nonlinear dynamical systems whose dynamics are unknown butsampled trajectories are available. One consequence is that theassessment works on a limited number of observations and doesnot require repeated simulations to achieve pre-definedstatistical properties of the assessment itself.Optionally, the reliability of the system may be determined whenthe at least one convex body and a pre-defined undesired stateare separate. The pre-defined undesired state is represented inone or more parameters, at least one of which is also aparameter detected (directly, or derived from detectedparameters, as described above) and recorded during operatingthe system according to the present method. Consequently, theirpre-defined undesired state is represented by one or moreranges, areas or (hyper–) volumes in the. If and only if theconvex body is entirely outside the pre-defined undesired state(meaning that the two are separate), the system is determined asreliable (with a residual risk, see above). In other words, theat least one convex body and the pre-defined undesired state arenon-intersecting; the volume of their intersection is zero. Thepre-defined undesired state does not have to be providedexplicitly in this format. The pre-defined undesired state mayalternatively be the inverse of a pre-defined desired state; oran intersection of a pre-defined undesired state and the inverseof a pre-defined desired state, for example.In one embodiment of the present method, determining at leastone convex body comprises: determining a center point of the atleast one cross-sectional; and determining a maximum distancebetween the center point and the pre-defined proportion of theparameter values of the cross-sectional in at least one pre-defined metric, wherein the convex body corresponds to the spacewithin the maximum distance around the center point. The atleast one pre-different metric, in case of two or moreparameters and parameter values, can be any metric or distancefunction in the mathematical sense, that is applicable in theparameter space in which the convex body is defined, such as forexample L1, L2, Lp, Linf.In this context, the center point may for example be determinedas the midrange value of each parameter from the at least onecross-sectional. The midrange value is relatively robust withrespect to the shape of the distribution of the respectiveparameter, thus providing a suitable reference also inapplications where – in general – different parameters havedifferent distributions.Optionally, the at least one pre-defined metric in thisembodiment can be a scaled distance function, wherein eachparameter is associated with a scaling factor, the methodfurther comprising: before determining the maximum distance,determining the scaling factors of the metric such that thedistance in Euclidian metric between the outermost of the pre-defined proportion of the parameter values of the cross-sectional and the convex body for each parameter is minimized.In other words, the convex body is skewed along each (parameter-) axis such that the minimum or maximum parameter value alongeach axis lies on the boundary of the convex body. The scalingfactors can be determined for each relative moment separately,resulting in a scaling time series.More generally, determining at least one convex body may moregenerally comprise: determining a convex body such that at leastone parameter value of the pre-defined proportion of the cross-sectional is a point of the boundary of the convex body.In one more specific embodiment, determining at least one convexbody comprises: determining a convex hull of the pre-definedproportion of the cross-sectional as the convex body. In thiscase, the shape of the convex body reflects and closely “wraps”and encloses the pre-defined proportion of the cross-sectional.The volume of the convex body so defined is generally smallerthan the volume of any other convex shape enclosing the samepre-defined proportion of the cross-sectional. consequently, theconvex body defined as the convex hull has the least likelihoodof intersecting any of the undesired states. An assessment basedon the convex hull is convex body thus provides a natural limiton the reliability, safety, and robustness of the system undertest.In this context, determining at least one convex body mayfurther comprise: determining a center point of the at least onecross-sectional; and determining a metric where every point ofthe boundary of the convex body has the same distance to thecenter point. In other words, the convex body is a sphere withunit radius in the metric determined in this way. Therefore, theconvex body can be scaled by adjusting the radius of the unitsphere in this metric, which generally translates to differentparameter-specific scaling in Euclidian space. The space orbasic metric used for finding the metric is irrelevant, becausethe definition relies not on specific or absolute distances, buton relative distance equality.In any embodiment defining a center point, the method mayinclude scaling the convex body with a buffer value from thecenter point to determine the at least one convex body. Forexample, when the convex body initially corresponds to the spacewithin the maximum distance around the center point, that convexbody can be scaled with the buffer value before using theresulting (scaled) convex body to determine the reliability ofthe system based thereon.For example, the buffer value may be a function of the nearestneighbors among a pre-defined proportion of initial states ofthe state time series.According to one embodiment of the present method, the buffervalue is determined at each moment based on a desired maximumresidual risk of the verification. By increasing the buffervalue, the residual risk can be decreased and vice versa.For example, the buffer value can be determined based on adifference of distances at each moment of the state time series,wherein the difference of distances is determined based on themaximum difference between the distances to the center of anyparameter values of a pre-defined proportion in the cross-sectional to its nearest neighbors within the initial states towhich the parameter values belong. The difference of distancesmay be between nearest neighbors within the pre-definedproportion of the cross-sectional or it may be a differencequotient relating distances within the cross-sectional todistances of associated initial states. In another example, thebuffer value may be determined based on the distances of theparameter values to the pre-defined unsafe states. In that case,while reliability will be found, high significance and / orexceeding residual risk may invalidate a positive assessment ofthe verified system.Determining the reliability of the system with the presentmethod may for example comprise determining a minimum distancebetween the center point and a pre-defined undesired state in ametric corresponding to the convex body; and determining basedon the minimum distance whether the at least one convex body andthe pre-defined undesired state are separate. This approachprovides a metric-agnostic assessment of reliability andprovides a relatively efficient approximation of what isgenerally a collision-detection.For long-running systems, i.e. systems running for significantdurations with ongoing, for example stabilizing, control, themethod defined above can be specialized when determining the atleast one cross-sectional comprises: determining a cross-sectional of each moment of the state time series; anddetermining at least one convex body comprises: determining areachable time series corresponding to the state time series,wherein each moment of the reachable time series has anassociated convex body, wherein the convex body of each momentis based on at least the pre-defined proportion of the cross-sectional of that same moment; and determining the reliabilityof the system comprises: determining the reliability of thesystem based on the convex body at every moment of the reachabletime series. The sampling rate of the time steps when recordingtraces can be directly dependent on the maximum possiblefrequency of the tested system and preferably at least doublethat frequency.In this context, the reliability of the system may be determinedbased on the pre-defined undesired state and the convex body atevery moment of the reachable time series being separate. If atany moment of the reachable time series, the convex body at thatmoment touches or intersects the pre-defined undesired state,reliability is neglected.The reachable time series may also be understood as astatistical reach tube. The statistical reach tube holds forvalues inside the initial distribution and with a specificsignificance (alpha) and residual risk (p). In other words, theapproach provides a statistical guarantee that the system willstay within desired (not undesired) boundaries with a certainprobability – meaning (at least) that the system is reliable –under the premise that the initial distribution (i.e. thedistribution of the initial states) is correct. Moreover, theposition and shape of the reach tube – if it is constructed fromproperties of the initial and recorded states alone – provides ameans to quantitatively assess the distance from undesiredstates, which can further inform the relevance of the foundreliability.As a particular use of the method disclosed above, the presentinvention also concerns a method for determining operationallimits of a cyber-physical system being controlled by acontroller, the method comprising: operating the system underdifferent operating conditions; and under each operatingcondition, determining the reliability of the system accordingto the method according to one of the variations described aboveto identify the operating condition as nominal operatingcondition in case the system is found to be reliable orotherwise as out-of-bounds operating condition. The differentoperating conditions are conditions specific to the applicationof the system, i.e., different environmental conditions, whichmay not be represented in the initial state. This may be thecase when the operating conditions are outside the control ofthe method and / or their distribution is unknown. In that case,this use allows to define a limited (although not unnecessarily)distribution within which the system operates reliably.In this context, the different operating conditions may forexample be different constraints for one or more controlvariables determined by the controller. In other words, theoperating conditions may also include configuration parametersof the controller. Those usually have no inherent distribution,unlike initial states. By varying them as operating conditions,their effect on the reliability can be assessed quantitatively.In one particular embodiment of this use, the differentoperating conditions (or parts of the different operatingconditions) may be simulated by applying different levels ofnoise to the detected parameter values, or to the initialstates, or to environmental conditions or configurationparameters of the controller, to determine whether thedetector(s) together with the controller are accurate enough forsafe operation. The noise may be generated and derived from anentropy source or a deterministic random number generator. Thenoise may optionally be sampled from a pre-defined distributionof optionally dependent random variables. In some cases it maybe random noise consisting of random samples (independentobservations) from a normal distribution with a pre-definedconstant mean and standard deviation. For example, by recreatingthe distribution of differences in the output of two differentsensors for the same fundamental parameter, the effects onreliability of replacing one sensor with the other can beevaluated before performing the actual replacement. Thisapproach provides a hybrid between testing the actual system(most realistic behavior) and using limited simulation toemulate system modifications and assess their impact onreliability in advance.Referring now to the drawings, wherein the figures are forpurposes of illustrating the present invention and not forpurposes of limiting the same:Fig. 1 schematically illustrates a first exemplary use caseof the present disclosure in the control of an invertedpendulum; Fig. 2a-d schematically illustrate different states of theinverted pendulum according to Fig. 1;Fig. 3a and 3b illustrate the reachable space (or set) forthe controlling a cartpole swingup of the inverted pendulum;Fig. 4 schematically illustrates a second exemplary usecase of the present disclosure on the control of a lane keepingvehicle; Fig. 5 schematically illustrates a third exemplary use caseof the present disclosure on adaptive cruise control of avehicle.In general, the method for verification, in particularstatistical verification, of a cyber-physical system beingcontrolled by a controller as presently disclosed follows somegeneral steps: The cyber-physical system to be verified isoperated starting from multiple different initial states.During that operation, a parameter value of at least oneparameter describing the state of the system is detected. Thisdetection can be repeated several times at different moments intime, resulting in sampled data. For (that is, beginning with)each of the initial states, the detected parameter values(samples) are recorded at corresponding relative moments overtime as a state time series. During operation, preferablyparameter values describing safety-relevant states of the systemare detected (or measured with sensors). Let us define, for thepurpose of describing a first embodiment, as a function of these parameter values at time tj starting atvalue x. Let us denote with fX the distribution of the states ofthe system.In the exemplary use case illustrated in Fig. 1-3 we start acartpole 1 of an inverted pendulum. The cartpole 1 is controlledvia video input with different light conditions. The initialstates are defined by different lux values (n = 1). Duringoperation we measure position and velocity of the cart and angleof the pole. These state variables are input to a controller 2.The controller 2 provides as control variable and system input(for use by the actuator of the cart) the velocity 4 of the cart3 along its axis of movement 5.Let us define with y the cart position, the pole angle and vthe measured cart velocity at time tj and: and thus d = 3 parameters of the system, with f(x,tj)irepresenting the recorded parameter values, i.e. the state timeseries.The sampling rate of time steps when recording traces isdirectly dependent on the maximum possible frequency of thetested system. Let fmax be the maximum possible frequency of agiven system. According to the Shannon-Nyquist Theorem, using asampling time ofwill be sufficient to detect all system changes. In other words,by sampling at this rate, the obtained traces will fully capturethe system’s behavior.For better readability we will drop the time index j when it isclear from the context that we are talking about a specific timestep. With X ∼ fX being a n-dimensional random variable (thisreflects the dimensionality of the possible differences in theinitial conditions) and our samples x ∈ V being i.i.d from thatdistribution.We sample additional N samples from the same distribution (wherea sample refers to a run of the system) and call the set ofthese samples V2 (it would be also possible to sample less ormore than N samples for the second set V2). Alternatively, if thesamples are provided without the possibility for additionalruns, we sample (without replacement) a pre-defined percentage(e.g. 50%) of V, call it V2 and remove these samples from V.At least one cross-sectional is determined comprising theparameter values of the same relative moment in all of the statetime series: Time series without a sample at the common relative moment ofthe cross-sectional are not included (alternatively: may beinterpolated between earlier and later samples of the respectivetime series).At least one convex body is determined based on a pre-definedproportion of the at least one cross-sectional through thefollowing steps in this example:1. Calculate for every time step the center point as themidrange value of all F(tj) being the function of themeasured values. With i ∈ {1, ... , d} being thedimensional index of a specific sample, j ∈ {0, ... , k}being the time step: with xc being the center at time tj. Multiple convex bodiescan be similarly constructed, each at one time tj,effectively creating a reach tube around the recordedparameter values over time.Choose desired metric Mj and calculate distance dj(·) anddistance differences between each point and its nearestneighbour λj(·) for all values: Before calculating and , we remove outliers fromthe set V2. This may be done either using IQR1.5 or - whenknowing the undesired states beforehand - by removing fromV2 the trajectories leading to undesired states.The metric Mj is a weighted or a weighted metric.With weighted metric we describe the following: the weightsare written inside a matrix and the metric definedas or . The same holds for the metricof the initial states Ms, which can be also chosen dependingon the distribution fX of the starting conditions. For easeof description in the present example, we will use Mj = Ms =M2 (only balls in Euclidean metric). While the entire cross-sectional F may in principle be used, using only partsreduces the impact of outliers / uncommon conditions.3. X ∼ fX is a random variable as defined above. Pick basedon and based for the ease ofimplementation, let (with thesechoices we are able to trade-off between tightness of thetube and confidence). For every time step, use astatistical method (e.g. clopper-pearson test; see CharlesJ Clopper and Egon S Pearson. The use of confidence orfiducial limits illustrated in the case of the binomial.Biometrika, 26(4):404–413, 1934.) and create a confidenceinterval to find p, so that holds with significance α. Use the samples from V toperform these tests.Alternatively, a confidence interval can be created to findp, so that: 4. Set tube radii toWe construct the tube at every time step tj as a d-dimensional ball in metric Mj with center xc(tj) and radiusδj. With significance α this method gives an upper bound pof the residual risk (= probability of being outside theconstructed tube) for values X ∼ fX.This reach tube can be used to check for intersections withunsafe or undesired states to assess the failure rate of thecyber-physical system or, specifically, its controller.Undesired states are given by convex areas in the d-dimensionalspace. These convex areas can also be time-dependent. In thepresent cartpole example we had defined with y the position ofthe cart on the rail (the length of the rail being 1.8, with theorigin of y in the center), v the speed of the cart and φ thepole angle, then the undesired states can be defined as (a stateis undesired if at least one of the conditions is true):The check of intersections with undesired states will beexecuted for every time step independently. For a check at giventime tj the tube will be translated for its center to be at theorigin of the parameter space. After that the weighting Matrix Aof metric Mj will be used to transform the tube into a ball andto transform the undesired states into that same space. Thattransformation happens by multiplication with A from the leftside (y = Ax with x being the tube or an undesired state).Finally, the reliability of the system is determined based onthe at least one convex body. The undesired states will still bedefined by a convex area after these transformations and theirdistance to the center can be compared to the radius of thetransformed tube. If their distance is smaller than this radiusthe systems safety can not be guaranteed.For example, regarding the cartpole use-case shown in Fig. 1-3,a neural network controller 2 is used to swingup and stabilizethe pole 5. The controller 2 together with the cartpole for acyber-physical system 15. This procedure is operated 10.000times. The recorded results are used in the method laid outabove to provide a guarantee that it is safe for untested cases:with significance α a statistical reach tube with residual risk<= p (probability that a trajectory will not be enclosed by thattube) is built. Fig. 3a and Fig. 3b show a few recordedtrajectories 7 as functions of time tj and a reach tube of convexbodies 8 constructed from the recorded trajectories 7 asvertical lines at each point in time tj, wherein Fig. 3a showsthe cosine of the pole angle φ and Fig. 3b shows the position yof the cart 3 along the axis of motion 5. Based thereon it showsthat there is no intersection of computed reach tube withundesired states (see above).In a second exemplary use case illustrated in Fig. 4 the methodis applied to a lane keeping task. A NN controller is used toautonomously drive a car 9 with the goal to perform lanekeeping. The car 9 and the controller (not shown) are a cyber-physical system 15. The question is how sensitive this NNcontroller reacts to different lighting conditions. A car 9 isdriven on a test track during different lighting conditions andthe distances 16, 17 of the car tires to the lanes 10, 11 aremeasured and saved as time series data indicated as a fewexemplary trajectories 12 (the distribution of the lightingconditions of this data reflects the real distribution ofdifferent lighting conditions during operation, for example byoperating the system at different times during the day). The cartires should not touch the lanes 10, 11 (touching them isdefined as undesired state). A statistical reach tube is builtaround time series data with measured distances to the lanes(residual risk <= p that a trajectory will not be enclosed bythat tube, with significance α). The present method shows thatthere is no intersection of computed reach tube with the lanes10, 11 (unsafe states).In a third exemplary use case illustrated in Fig. 5 a NNcontroller adjusts a car’s speed (ego car, E) automatically tomaintain a safe following distance to another vehicle (lead car,L) and stay within the speed limit. The ego car E and itscontroller are the cyber-physical system 15. The question is howsensitive this NN controller reacts to different distances 13between the ego car E and the lead car L as well as varyingspeed differences. Two cars E, L are driven on a test track andthe speed of the second car (ego car) is being adjusted by theNN controller. Every run is initialized with different positionsand speed of the lead car L and the ego car E (the distributionof the initialization conditions of this data – the initialstates of operation – reflects the real distribution ofdifferent conditions during operation). Defined unsafe statesare when the distance to the lead car is too small (not onlyabsolute distance but also relative to time gap and ego carvelocity). A reach tube is built around the time series data(schematically indicated as corresponding trajectories 14)according to the present disclosure with measured distances 13between the cars E, L, time gap and ego car velocity as systemparameters (residual risk <= p that a trajectory will not beenclosed by that tube, with significance α). Safety: show thatthere is no intersection of computed reach tube with the minimumrequired distance (unsafe states) and the maximum allowed speedlimit.For another exemplary use case we consider a general cyber-physical system with a NN controller getting the input via asensor. The systems goal is to not reach an undesirable state.Switching the sensor might lead to slight changes in the inputof the controller, due to different sensor accuracy andprecision for example. The distribution of these precisiondifferences is known (e.g. the error of sensors are normallydistributed with known mean and standard deviation). If thedistribution is not known, we measure the differences of sensoroutputs and use a statistical test (e.g. KS-Test) to check forthe underlying distribution. The question is whether one canverify the systems safety for different sensors. The presentmethod allows for this by adding for every recorded trace a biasto the input of the controller to simulate another sensor beingused. The used sensor in combination with the biases shallrepresent the distribution of the precision differences.According to the present method, a statistical reach tube isbuilt around the resulting time series data (residual risk <= pthat a trajectory will not be enclosed by that tube, withsignificance α) with the sensor bias being an additionaldimension of the safety-relevant states of the system. Using thepresent method we can show that there is no intersection ofcomputed reach tube with the undesired states.

Claims

Claims:

1. A method for verification of a cyber-physical system (15)being controlled by a controller (2), the method comprising:- operating the system (15) starting from multipledifferent initial states;- detecting during operation a parameter value of at leastone parameter describing the state of the system;- recording the parameter values at corresponding relativemoments over time as a state time series (7) for each of theinitial states;- determining at least one cross-sectional comprising theparameter values of the same relative moment in all of the statetime series;- determining at least one convex body (8) based on a pre-defined proportion of the at least one cross-sectional; and- determining the reliability of the system (15) based onthe at least one convex body (8).

2. The method according to claim 1, characterized in that thereliability of the system (15) is determined when the at leastone convex body (8) and a pre-defined undesired state areseparate.

3. The method according to claim 1 or 2, characterized in thatdetermining at least one convex body (8) comprises:- determining a center point of the at least one cross-sectional; -determining a maximum distance between the center pointand the pre-defined proportion of the parameter values of thecross-sectional in at least one pre-defined metric,wherein the convex body (8) corresponds to the space withinthe maximum distance around the center point.

4. The method according to claim 3, characterized in that thecenter point is determined as the midrange value of eachparameter from the at least one cross-sectional.

5. The method according to claim 3 or 4, characterized in thatthe at least one pre-defined metric is a scaled distancefunction, wherein each parameter is associated with a scalingfactor, the method further comprising:- before determining the maximum distance, determining thescaling factors of the metric such that the distance inEuclidian metric between the outermost of the pre-definedproportion of the parameter values of the cross-sectional andthe convex body (8) for each parameter is minimized.

6. The method according to claim 1 or 2, characterized in thatdetermining at least one convex body (8) comprises:- determining a convex body (8) such that at least oneparameter value of the pre-defined proportion of the cross-sectional is a point of the boundary of the convex body (8).

7. The method according to claim 6, characterized in thatdetermining at least one convex body (8) comprises:determining a convex hull of the pre-defined proportion ofthe cross-sectional as the convex body (8).

8. The method according to claim 6 or 7, characterized in thatdetermining at least one convex body (8) further comprises:- determining a center point of the at least one cross-sectional; -determining a metric where every point of the boundary ofthe convex body (8) has the same distance to the center point.

9. The method according to any one of claims 3 to 5, or 8,characterized in that determining at least one convex body (8)comprises: -scaling the convex body (8) with a buffer value from thecenter point.

10. The method according to claim 9, characterized in that thebuffer value is a function of the nearest neighbors among a pre-defined proportion of initial states of the state time series.

11. The method according to claim 9 or 10, characterized inthat the buffer value is determined at each moment based on adesired maximum residual risk of the verification.

12. The method according to any one of claims 9 to 11,characterized in that the buffer value is determined based on adifference of distances at each moment of the state time series,wherein the difference of distances is determined based on themaximum difference between the distances to the center of anyparameter values of a pre-defined proportion in the cross-sectional to its nearest neighbors within the initial states towhich the parameter values belong.

13. The method according to any one of claims 3 to 5 or 8 to12, characterized in that determining the reliability of thesystem (15) comprises:- determining a minimum distance between the center pointand the pre-defined undesired state in a metric corresponding tothe convex body (8);- determining based on the minimum distance whether the atleast one convex body (8) and the pre-defined undesired stateare separate.

14. The method according to any one of claims 1 to 13,characterized in that determining the at least one cross-sectional comprises:- determining a cross-sectional of each moment of the statetime series (7); anddetermining at least one convex body (8) comprises:- determining a reachable time series corresponding to thestate time series (7), wherein each moment (tj) of the reachabletime series has an associated convex body (8), wherein theconvex body (8) of each moment (tj) is based on at least the pre-defined proportion of the cross-sectional of that same moment(tj); anddetermining the reliability of the system (15) comprises:- determining the reliability of the system based on theconvex body (8) at every moment (tj) of the reachable timeseries.

15. The method according to claim 14, characterized in that thereliability of the system (15) is determined based on the pre-defined undesired state and the convex body (8) at every momentof the reachable time series being separate.

16. A method for determining operational limits of a cyber-physical system (15) being controlled by a controller (2), themethod comprising:- operating the system (15) under different operatingconditions; -under each operating condition, determining thereliability of the system (15) according to the method of anyone of claims 1 to 15 to identify the operating condition asnominal operating condition in case the system is found to bereliable or otherwise as out-of-bounds operating condition.

17. The method according to claim 16, characterized in that thedifferent operating conditions are different constraints for oneor more control variables determined by the controller (2).

18. The method according to claim 16, characterized in that thedifferent operating conditions are simulated by applyingdifferent levels of noise to the detected parameter values or tothe initial states to determine whether the detector togetherwith the controller (2) are accurate enough for safe operation.

Images