CONDITION ASSESSMENT OF A SYSTEM

Description

[0001] The present invention relates to a computer-implemented method for assessing the condition of a technical system, a condition assessment device for assessing the condition of a technical system, and a computer program.

[0002] Many properties of a system, especially non-functional properties, are defined discretely, often in binary terms. For example, whether a system fulfills an essential function, such as being safe, or not, such as being unsafe. The failure to fulfill such properties can have various adverse effects depending on the specific system and / or application. Therefore, trend analysis and subsequent forecasting of the future development of a binary or discrete quantity with few possible values ​​is often not feasible.

[0003] To determine so-called RAMS (reliability, availability, maintainability, and safety), such as reliability or availability, reference is made, for example, to the correct functioning of the system at a given time. A system either functions correctly, meaning it works as intended, or it does not, meaning it does not work as intended.

[0004] Characteristic values ​​determined on the basis of binary states or discrete states with few values ​​are difficult to predict, since the change of state occurs abruptly and suddenly.

[0005] For example, if the reliability of the system is required over a defined period, it is desirable to be able to predict a change in state from "system working" to "system not working." This would make it possible to take timely and targeted corrective action, for example, through appropriate measures such as replacing components, updating software, transferring functionality to alternative systems, and so on.

[0006] Stochastic approaches such as fault trees or Markov chains can be used to estimate the probability of a specific event occurring within a defined future timeframe. For example, the expected availability of a system can be calculated based on the fault relationships leading to system failures and fault data related to the underlying events. However, the results of such calculations are purely stochastic and do not take into account the actual state of the system, which is crucial in determining whether the event actually occurs.

[0007] The German patent application DE 10 2017 200 544 A1 concerns a method for determining the lead time for replacing an optical smoke detector. A current value for the degree of soiling of the optical smoke detector, as well as subsequent values ​​for the degree of soiling, are determined. The lead time is calculated based on the previous values ​​for the degree of soiling using a trend analysis.

[0008] The publication EP 2 950 176 A1 discloses a method for generating an event message. An automation component monitors at least one state parameter. At least one limit value is defined for this state parameter. If the limit value is violated, the event message is generated.

[0009] The publication EP 3 945 383 A1 discloses a method for monitoring and controlling a continuous flow motor. The method comprises collecting several current states of at least one filter over a period of at least three days. The collected states are used to monitor the at least one filter.

[0010] It is an object of the present invention to provide a means of assessing the state of a technical system that can be characterized by a discrete state parameter, so that in particular an improved, and especially a more accurate, prediction regarding the discrete state parameter becomes possible.

[0011] This problem is solved by the respective subject matter of the independent claims. Advantageous further developments and preferred embodiments are the subject matter of the dependent claims.

[0012] The invention is based on the concept of considering, in addition to a discrete first state parameter that can assume N different discrete values, a second state parameter for characterizing the system. This second state parameter is either a continuous state parameter or can assume M different discrete values, where M is greater than N. The second state parameter is chosen such that the first state parameter can be uniquely derived from it. Based on a measurement of the system, the second state parameter is determined, and depending on this, as well as on a critical value for the first state parameter, an evaluation parameter for assessing the state of the technical system is calculated.

[0013] According to one aspect of the invention, a computer-implemented method according to claim 1 is specified.

[0014] Unless otherwise specified, all steps of the computer-implemented method can be performed by at least one computing unit, which can also be referred to as a data processing device. In particular, the data processing device, which comprises at least one processing circuit configured or adapted to carry out a computer-implemented method according to the invention, can perform the steps of the computer-implemented method. For this purpose, a computer program can be stored in the data processing device, in particular one containing instructions which, when executed by the data processing device, in particular the at least one processing circuit, cause the data processing device to execute the computer-implemented method.

[0015] Each embodiment of the computer-implemented method according to the invention for assessing the condition of a technical system directly implies a method for assessing the condition of a technical system that is not fully computer-implemented and which, in addition to the steps of the computer-implemented method, also includes generating the measurement result by carrying out at least one measurement on the system, in particular by means of a measuring system.

[0016] The technical system can, for example, comprise a machine or plant, or several physically, virtually, and / or functionally interconnected machines and / or plants. Exemplary, but not limited, application areas include systems in the field of energy technology, such as plants and / or machines for energy generation, energy conversion, and / or energy transmission. Further non-limiting application areas lie in the field of mobility, such as rail transport, where the system can include, for example, train components, locomotives, track systems or parts thereof, passenger cars, trucks, and so on; and in the field of industrial production, where the system can include, for example, production machines or plants, manufacturing machines or plants, test equipment, monitoring systems, conveyor machines or plants, process engineering plants, and so on.Further, non-restrictive areas of application lie in the field of medical technology, so that the system can include, for example, devices for medical imaging, such as MRI systems, X-ray-based imaging systems like CT systems, ultrasound-based imaging systems, PET systems, and so on. The system can also include one or more robotic systems.

[0017] Generally, a system includes one or more hardware components as well as one or more software components. However, it is also possible for the system to consist of only one or more hardware components or only one or more software components.

[0018] The fact that the system can be characterized by the first state parameter can be understood to mean that the first state parameter can be used to characterize the system. An overall state of the technical system can potentially be characterized by a multitude of state parameters, whereas the first state parameter merely characterizes a specific quality or property, in particular a non-functional property, of the system. The same applies analogously to the second state parameter. The first and second state parameters are not independent of each other, since one of the N discrete values ​​for the first state parameter can be uniquely calculated from the second state parameter. Conversely, the second state parameter cannot generally be uniquely derived from a given value for the first state parameter.This is reflected in particular in the fact that the number of values ​​that the second state parameter can take is greater than the number N of values ​​that the first state parameter can take.

[0019] In other words, the first and second state parameters can relate to similar or identical qualities or properties of the technical system, but the resolution—that is, the number of values ​​that can be used to describe the quality or property of the system—differs depending on the state parameter used. Thus, using the second state parameter increases the resolution with which the quality or property of the technical system can be characterized compared to using the first state parameter.

[0020] The fact that the first state parameter can take on exactly the N discrete values ​​can be understood in particular as meaning that a range of values ​​of the first state parameter consists of the N discrete values, i.e., the first state parameter cannot take on any values ​​other than the N discrete values.

[0021] If the second state parameter can take on all values ​​within the continuous range, this can be understood in particular to mean that the second state parameter cannot take on values ​​that lie outside the continuous range.

[0022] The continuous range of values ​​can be represented, for example, as the interval [m0, m] or as the interval [m0, m[, or as the interval ]m0, m] or as the interval ]m0, m[, where m0 and m are real numbers and, for example, m0 = 0 and m > 0.

[0023] If the second state parameter can take on exactly the M discrete values, this can be understood in particular to mean that a range of values ​​of the second state parameter consists of the M discrete values, i.e., the second state parameter cannot take on any values ​​other than the M discrete values.

[0024] In order to generate the measurement result, at least one measurement is carried out on the system, for example by means of a measuring system.

[0025] As mentioned above, the current value of the second state parameter can be determined directly from the measurement result. For example, the second state parameter can be measured directly as a measurand. Alternatively, at least one measurand of the system can be measured, where the second state parameter is not one of the measurands, and the current value of the second state parameter can be calculated from the measured measurand, thus resulting in an indirect determination of the second state parameter from the measurement result.

[0026] The evaluation parameter can be understood as the value of a predefined evaluation parameter or evaluation function. Depending on the current value of the second state parameter and the predefined first critical value, the evaluation function can, for example, be evaluated to determine the evaluation parameter. Accordingly, the evaluation function, and thus the evaluation parameter, relates the second state parameter to the first critical value of the first state parameter. In this way, it becomes possible to estimate the proximity of the system state to the state defined by the first critical value by determining the current value of the second state parameter as described.Determining the current value of the first state parameter may not be straightforward or may be of little value, as it only provides information about whether the first critical value has been reached, especially if N = 2. However, even for N greater than 2, considering the second state parameter allows for a more accurate estimate of how close the system's state is to the first critical value, since M > N or the second state parameter is even a continuous state parameter, taking into account appropriate tolerances and / or measurement uncertainties.

[0027] The present invention thus compensates for the limitation of the first state parameter to two or a few discrete values, in particular N values, and the associated limited suitability for assessing the state of the system by providing a more accurate reference to the state of the system corresponding to the first critical value via the second state parameter than would be possible solely by determining or estimating the first state parameter.

[0028] This can be illustrated particularly clearly using the example of N = 2. The first state parameter can then be either 0 or 1. If the first critical value is 0, for example, determining the current value of the state parameter only reveals whether the critical value has been reached or not. However, if the second state parameter is allowed a continuous range of values ​​from 0 to 1, where, for instance, the first state parameter is 0 when the second state parameter is 0, and the first state parameter is 1 when the second state parameter is 1, then determining the second state parameter allows us to estimate the probability that the first state parameter is currently, at a specific future time, or within a specific future period.

[0029] If the second state parameter is determined to have a value of 0.9, a different prediction for the first state parameter might be possible than if the second state parameter has a current value of 0.1. For example, in the case of 0.9 for the second state parameter, the probability of the first state parameter becoming 0 within a given period might be lower than for a value of 0.1 for the second state parameter.

[0030] This principle can of course also be applied to other values ​​for N and to other continuous or discrete ranges of values ​​for the second state parameter, whereby M > N always holds true. The example explained is merely to be understood as a non-limiting application case in which the advantages of the invention are particularly easy to recognize.

[0031] The evaluation parameter can be used, for example, to forecast the state of the system, particularly regarding a future value for the first state parameter. Alternatively or additionally, the evaluation parameter can also be used as a basis for deciding whether to initiate a measure on the system, such as whether to adjust operating parameters, perform a software update, carry out maintenance, or replace the system or its components, and so on.

[0032] According to at least one embodiment of the computer-implemented method, 1 < N ≤ 5. Preferably, 1 < N ≤ 3, more preferably, N = 2.

[0033] According to at least one embodiment, the second state parameter can take on all values ​​in the continuous range of values, or M ≥ 5*N, preferably M ≥ 10*N, in particular M ≥ 50*N or M ≥ 100*N.

[0034] According to at least one embodiment, the first critical value corresponds to a state of the system in which the system cannot meet a predetermined performance requirement and / or a predetermined reliability requirement and / or a predetermined safety requirement and / or a predetermined availability requirement.

[0035] If the first state parameter is equal to the first value of the N discrete values, for example, 0, then the system cannot meet the performance, reliability, safety, and / or availability requirements. In this case, the first value, for example, 0, corresponds to the first critical value. If the first state parameter has a second value different from the first value, for example, 1, then this corresponds to a state in which the system can meet the performance, reliability, safety, and / or availability requirements. If N > 2, then all other values ​​of the N discrete values, except for the first critical value, also correspond to a situation in which the system can meet the stated requirement and may even exceed it, or meet additional requirements, or the like.

[0036] According to at least one embodiment, a second critical value for the second state parameter is determined depending on the first critical value. The evaluation criterion is determined depending on the distance between the current value of the second state parameter and the second critical value.

[0037] The distance can be determined, in particular, by a suitable distance metric. Specifically, the distance can correspond to a difference between the current value of the second state parameter and the second critical value, or to an absolute value of this difference.

[0038] Since every value for the second state parameter uniquely corresponds to a corresponding value for the first state parameter, the first and second critical values ​​are also related in this way, such that the first critical value for the first state parameter can be uniquely derived from the second critical value for the second state parameter. However, this does not mean that the second critical value can also be uniquely determined from the first critical value.

[0039] The second critical value is therefore determined in particular depending on the first critical value and a predefined rule. For example, the second critical value can correspond to the smallest possible value for the second state parameter that leads to the first critical value for the first state parameter, or to the largest such value for the second state parameter, or to an average of all values ​​for the second state parameter that lead to the first critical value, or something similar.

[0040] In the following example use case, let N = 2, where 0 corresponds to the first critical value and 1 to a non-critical value for the first state parameter. The second state parameter can, for example, take any integer value from 0 to M, where M can be arbitrarily large, for example, 10,000. The first critical value of 0 could, for example, correspond to a situation in which the system cannot process 1,000 requests within a given maximum time. Therefore, if the system can process 1,000 or more requests within the given time, the first state parameter is 1; otherwise, it is 0. The second state parameter could then, for example, correspond to the actual number of requests that the system can process within the given maximum time. If this number is 1...If the number of requests that can be processed within the specified maximum time is 000 or greater, then it follows that the first state parameter is equal to 1; if the number of requests that can be processed within the specified maximum time is 999 or less, then it follows that the first state parameter is equal to 0.

[0041] The second critical value can then be defined, for example, as 999, which corresponds to the largest value for the second state parameter that leads to the first critical value of 0 for the first state parameter. However, other options for choosing the second critical value are also possible, such as 1200, to ensure a certain minimum distance from the first critical value.

[0042] According to at least one embodiment, the first state parameter assumes the first critical value if and only if the second state parameter is less than or equal to the second critical value.

[0043] The second critical value is in particular smaller than M or smaller than m, especially when m0 = 0.

[0044] Alternatively, the first state parameter can assume the first critical value if and only if the second state parameter is greater than or equal to the second critical value.

[0045] Which of the alternatives is advantageous depends on the specific design of the system and its functionalities.

[0046] According to at least one embodiment, the second state parameter quantifies a performance of the system, with the performance of the system being greater the larger the second state parameter is.

[0047] In such embodiments, for example, the first state parameter can correspond to the first critical value if and only if the second state parameter is less than or equal to the second critical value.

[0048] Performance can be assessed differently depending on the system's design. For example, performance can be defined by how quickly the system can perform a specific task, how often it can perform such a task within a given timeframe, with what accuracy the system can perform a specific task, and so on.

[0049] According to at least one embodiment, a forecast for the first state parameter is carried out depending on the evaluation parameter.

[0050] The forecast for the first state parameter can be understood in particular as an estimate of what the value for the first state parameter will be in a future period or at a future point in time, or how the value for the first state parameter is expected to develop in the future period.

[0051] According to at least one embodiment, the time course of the evaluation parameter is determined by repeatedly performing the steps of obtaining the measurement result, optionally performing at least one measurement on the system, determining the current value for the second state parameter, and determining the evaluation parameter during an evaluation period, for example, periodically or cyclically. The forecast for the first state parameter is performed based on the time course of the evaluation parameter.

[0052] This allows for a more accurate and / or reliable prediction of the first state parameter.

[0053] According to at least one embodiment, depending on the evaluation parameter, in particular the time course of the evaluation parameter, a probability is calculated that a current value of the first state parameter corresponds to the first critical value.

[0054] This can be particularly advantageous if the current value of the first state parameter cannot be measured, or if measuring the first state parameter would adversely affect the operation or function of the system, for example, by preventing the system from performing its intended function. For instance, during an operational phase of a system, it is either impossible or only possible with increased time expenditure and production downtime to conduct performance tests to determine the first state parameter. Therefore, in such embodiments, it is advantageous to use the second state parameter, which can be acquired, particularly automatically, during the normal operation of the system.

[0055] According to at least one embodiment, a probability is calculated, depending on the evaluation parameter, in particular the time course, that the first state parameter will assume the first critical value within a given future period. The calculation of the probability can, in particular, correspond to or be part of the forecast.

[0056] According to at least one embodiment, depending on the evaluation parameter, and in particular on the time course of the evaluation parameter, a point in time is estimated at which the first state parameter is expected to reach its first critical value. The forecast can therefore correspond to the estimation of the point in time or include the estimation of the point in time.

[0057] According to a further aspect of the invention, a method for condition assessment, in particular for automatic condition assessment, of a technical system is provided. In this method, a computer-implemented procedure according to the invention is carried out, and the measurement result is generated by performing at least one measurement on the system.

[0058] Depending on the system design, the measurement can be implemented in different ways. For example, to perform at least one measurement, the state of a buffer, such as a message buffer, a queue, or the like, memory utilization, throughput time, and so on, can be measured.

[0059] According to at least one embodiment, the at least one measurement is carried out fully automatically using a measuring system in accordance with a predetermined measurement protocol.

[0060] The measurement protocol can be understood as defining which measurement is carried out at what time, with what frequency, and so on.

[0061] According to another aspect of the invention, a condition assessment device for the condition assessment of a technical system is specified according to claim 13.

[0062] According to at least one embodiment of the condition assessment device, it has a measuring system which is configured to perform at least one measurement on the system in order to generate the measurement results and transmit them to the at least one computing unit.

[0063] Depending on the system's design, the measurement system can be configured differently. For example, the measurement system may include one or more sensors for measuring one or more measurands. Measuring a measurand can also involve reading a memory area, such as a buffer, using appropriate measuring instruments, which may be partially or fully implemented in software.

[0064] Further embodiments of the condition assessment device according to the invention follow directly from the various embodiments of the computer-implemented method and the condition assessment method according to the invention, and vice versa. In particular, the condition assessment device according to the invention can be configured or programmed to carry out a computer-implemented method or a method according to the invention, or the condition assessment device according to the invention carries out such a computer-implemented method or method.

[0065] According to another aspect of the invention, a computer program product is specified according to claim 15.

[0066] A computing unit can be understood, in particular, as a data processing device containing a processing circuit. The computing unit can therefore process data to perform arithmetic operations. This may also include operations to perform indexed access to a data structure, such as a lookup table (LUT).

[0067] The computing unit may, in particular, contain one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more systems on a chip (SoCs). The computing unit may also contain one or more processors, for example, one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The computing unit may also include a physical or virtual array of computers or other units of the aforementioned type.

[0068] In various embodiments, the computing unit includes one or more hardware and / or software interfaces and / or one or more storage units.

[0069] A storage unit can be volatile data storage, for example as dynamic random access memory (DRAM) or static random access memory (SRAM), or as non-volatile data storage, for example as read-only memory (ROM), programmable read-only memory (PROM), erasable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), flash memory or flash EEPROM, ferroelectric random access memory (FRAM), magnetoresistive random access memory (MRAM), or phase-change random access memory.It should be designed as PCRAM (English: "phase-change random access memory").

[0070] When, within the scope of the present disclosure, it is stated that a component of the condition assessment device according to the invention, in particular the at least one computing unit of the condition assessment device, is configured, designed, or the like to perform or realize a certain function, to achieve a certain effect, or to serve a certain purpose, this can be understood to mean that, beyond the basic or theoretical usability or suitability of the component for this function, effect, or purpose, the component is, through appropriate adaptation, programming, physical design, and so forth, actually and concretely capable of performing or realizing the function, achieving the effect, or serving the purpose.

[0071] Further features of the invention will become apparent from the claims, the figures, and the description of the figures. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown in the figures, may be encompassed by the invention not only in the combinations specified, but also in other combinations. In particular, the invention may also encompass embodiments and combinations of features that do not have all the features of an originally formulated claim. Furthermore, the invention may encompass embodiments and combinations of features that go beyond or deviate from the combinations of features set out in the cross-references to the claims.

[0072] The invention will be explained in more detail below with reference to specific exemplary embodiments and associated schematic drawings.

[0073] This shows FIG 1 a schematic block diagram of a system and an exemplary embodiment of a condition assessment device according to the invention; and FIG 2 a flowchart of an exemplary embodiment of a method according to the invention for assessing the condition of the technical system.

[0074] In the FIG 1 Figure 1 schematically shows an exemplary embodiment of a condition assessment device 2 according to the invention for the condition assessment of a technical system 1. The system 1 can be characterized by a predetermined first condition parameter, which can assume N different discrete values, where N is a natural number and greater than 1, preferably N = 2. Furthermore, the system 1 can be characterized by a second condition parameter, which can assume all values ​​in a continuous range or which can assume M different discrete values, where M is a natural number and greater than N, in particular much greater than N, for example at least 10*N or at least 100*N.

[0075] The condition assessment device 2 has a measuring system 4 which can perform one or more measurements on the system 1 to measure one or more measured quantities, and a computing unit 3 which is connected to the measuring system 4 in order to obtain a measurement result, in particular the measured values ​​for the at least one measured quantity.

[0076] The condition assessment device 2 is in particular designed to carry out an exemplary embodiment of a method according to the invention for the condition assessment of the technical system 1.

[0077] FIG 2Figure 1 shows a flowchart of such a method according to the invention for the condition assessment of the technical system 1. In step S1, at least one measurement is carried out on the system 1 using the measuring system 4, in particular fully automatically, according to a predefined measurement protocol, in order to generate the measurement result. In step S2, the measurement result is transmitted to the processing unit 3, and the processing unit 3 can carry out an exemplary embodiment of a computer-implemented method according to the invention for condition assessment, which is represented by steps S3 to S5.

[0078] The second state parameter and the first state parameter are related to each other in such a way that from every value that the first second state parameter can assume, one of the N discrete values ​​for the first state parameter can be uniquely calculated.

[0079] To execute the computer-implemented procedure, in step S3, the processing unit 3 determines a current value for the second state parameter based on the measurement result. Depending on this current value and a predefined first critical value from among the N discrete values ​​for the first state parameter, the processing unit 3 determines an evaluation parameter in step S4. Based on this, in step S5, the processing unit 3 can, for example, perform a prediction for the first state parameter.

[0080] Further details of exemplary embodiments of the invention are explained below. The first state parameter is referred to as NF1 and the second state parameter as NF2.

[0081] Based on a discrete definition NF1 for determining non-functional properties of system 1, the quantity NF2 can be derived, which can have values ​​in a larger number space than NF1 or values ​​in a continuous number space, and has, for example, the following properties: The original discrete definition of NF1 can be recovered from NF2. NF2 is determined by data from System 1 that are available for analysis. a. For example, at operating time, also referred to as OPS. Then, for example, the value of NF2 can be quantified at any given time. b. And / or at development time, also referred to as DEV, in different development environments, test stages, or integration stages. Then, an evaluation of the non-functional properties for this environment can be performed in the respective context. The absolute values ​​between the environments or from DEV to OPS are not necessarily to be interpreted identically. For each value of NF2, it is possible to assess how critical a system state of System 1 is with respect to NF1. This can be quantified by the evaluation function B or corresponding evaluation parameters B(NF2).The assessment criterion can, for example, indicate whether it is likely that NF1 will reach its first critical value in the near future, which can also be referred to as a "violation of NF1," or what the risk of this is. Both discrete assessments with a suitably large number of classes and continuous assessments are possible.

[0082] The data required for calculating NF2 and B(NF2) can, for example, be automatically collected to enable an automated response. Depending on the application, semi-automated data collection could also be implemented. For instance, a data collection and evaluation process could be started manually.

[0083] In other scenarios, purely manual data collection may suffice. However, this requires more effort, longer cycle times, and a greater risk of introducing errors into the measurement process through manual work compared to full automation.

[0084] The collected values ​​can then be fed back, for example via DevOps methods, to the department that performs an evaluation and may trigger further reactions.

[0085] In one example, in DEV, the measured variables for each relevant system state are automatically collected in a CI (continuous integration) process, and based on this, NF2 is calculated and the rating B(NF2) is determined. This data, along with information about the development stage and the data collection environment, is collected in a database.

[0086] In OPS, at relevant times, particularly by recording the exact system version of the software and / or hardware of System 1, and relevant boundary conditions from the system environment, NF2 and B(NF2) are also determined and transmitted back to DEV via DevOps methods. This data can also be collected in a database, for example, along with the information on the system version and / or boundary conditions from OPS.

[0087] Application-specific parameters can be captured in both DEV and OPS that are relevant for later analyses. Parameters can also be captured here for which no explicit relationship is known, in order to investigate whether statistical methods or machine learning (ML) methods, such as neural networks, can identify such relationships.

[0088] This collected data allows for observation of the effects of system changes. Trends for NF2 and B(NF2) in the respective DEV and OPS environments can be identified and visualized.

[0089] In principle, this database allows for the identification of relationships between NF2 and B(NF2) between system versions and system environments, especially between different environments in DEV and different environments in OPS.

[0090] Trends and correlations can then be derived both for a system state across different application environments, and for different system states within a single application environment.

[0091] In DEV, various integration and test environments are typically available, depending on the chosen integration and testing process. With a fixed version of System 1, the metrics required to determine NF2 can be collected as described. NF2 and B(NF2) can be calculated for each of these environments. Using the NF2 and B(NF2) data, the impact of NF2 and B(NF2) on OPS can be predicted or estimated. This can initially be done using heuristics. Over time, as the data set grows, machine learning methods can also be applied.

[0092] This method can be particularly useful in DEV (Development Design) before the release and deployment of a specific system version according to OPS (Operational Performance Standards). Trends in the various DEV development environments can be captured to enable a forecast of the system's performance in OPS. This approach can be used to complement established quality assurance measures.

[0093] Another exemplary application in DEV is the continuous use of the determined evaluation indicators to manage development activities. Unfavorable trends, such as a deterioration of the evaluation indicators, can be identified, the causes can be analyzed, and appropriate corrective measures can be planned.

[0094] As an example of an application in OPS, after a release or deployment, metrics for determining NF2 and B(NF2) can be collected in the OPS environments selected for deployment. This helps determine whether the release represents an improvement with respect to non-functional properties, is sufficient, or whether a rollback is necessary. It can also support the decision of whether a deployment to all, or to which additional, OPS environments is justifiable, thus contributing to the implementation of DevOps processes. The concrete, local observation of these values ​​in the various OPS environments where the release was rolled out enables local decisions specific to each individual OPS environment.

[0095] As the data base grows, it is also possible to learn about correlations in order to predict in which environments the rollout can be carried out without taking on undesirably high risks.

[0096] Preferably, the acquisition of the necessary metrics for determining NF2 is fully automated. Appropriate decision rules can then be defined for this purpose. The evaluation of trends, for example, can also be automated. Initially, a human-in-the-loop approach can be chosen as long as the sufficient quality of the decisions made by computing unit 3 is not yet assured. This makes the method usable in a DevOps context.

[0097] The continuous determination of NF2 and / or B(NF2) also enables retrospective analyses of which changes in system z1 lead to a positive or negative change in the evaluation parameters. This facilitates the analysis of causes, for example, in the case of a detected deterioration of relevant system qualities.

[0098] As another exemplary use case, consider a system 1 that should be able to process 1000 requests within a given time. For example, NF1 = 1 if the system 1 is sufficiently powerful at a given time to potentially process 1000 incoming requests; otherwise, NF1 = 0. In development, for example, a test with a corresponding request rate could determine whether the system can process 1000 requests within the given time, and what measurable effects this has on other system areas, such as buffer or queue filling.

[0099] In OPS, NF1 may not be directly measurable, for example if fewer than 1000 requests actually arrive within specified time periods.

[0100] NF2 can be the number of requests that System 1 can actually process within the given time. B(NF2) evaluates this in relation to the currently required performance, thus quantifying how close System 1 is to NF1 = 0 or how close to NF2 < 1000. In DEV, targeted performance tests, for example by increasing the request rate, could determine up to how many requests System 1 can process in a timely manner within the given time.

[0101] In OPS, it may not be possible to perform such a performance test during operation without risking that System 1 will not be able to process the actual pending requests in a timely manner. In this case, system parameters such as buffer or queue levels can serve as metrics to allow for an estimation.

[0102] For B(NF2), depending on the application, a distance metric to a second critical value for NF2, which corresponds to the first critical value for NF1, can be chosen, for example, to the critical threshold of required or expected requests in relation to the currently feasible number of requests.

[0103] The performance level of System 1, thus determined on a continuous scale or on a discrete scale with many levels (in contrast to the original binary scale or discrete scale with very few levels), can then be recorded over time. The collected data for the continuous evaluation of System 1's performance can be analyzed for trends or tendencies.

[0104] This allows, for example, the detection of changes over time with the same system version but different environmental conditions, and / or changes across different system versions with potentially the same or changed environmental conditions, and / or changes in the correlation between DEV performance and OPS performance with the same system version in different OPS environments.

[0105] Various embodiments of the invention allow a system property NF1, which is originally only detectable in binary terms, to be replaced by a continuous variable NF2. Based on this, it is also possible to determine how close the system 1 is to violating the binary system property NF1. In particular, trends and tendencies regarding the relevant non-functional properties can be identified early on. Monitoring over time in both DEV and OPS makes it possible to initiate necessary system adjustments or development activities when the characteristic values ​​approach critical ranges. This might not be possible without continuous evaluation in this form, since the discrete non-functional property NF1 is still fulfilled.

[0106] The necessary data for determining the continuous variable NF2 can be acquired automatically. This makes corresponding implementations of the method applicable in automated processes such as CI and DevOps.

[0107] The automated acquisition of underlying system properties at various development stages, system versions, and system environments, as well as the parameters NF1, NF2, and B(NF2), particularly enables the application of machine learning. For example, this allows for the learning of DEV forecasting based on OPS, which, given a sufficient data basis, can achieve better results compared to heuristics.

Claims

1. Computer-implemented method for assessing the state of a technical system (1) characterizable by a predetermined first state parameter and characterizable by a predetermined second state parameter, - the first state parameter being able to assume precisely N different discrete values, N being a natural number and greater than one; - the second state parameter being able to assume all values in a continuous value range or being able to assume precisely M different discrete values, M being a natural number and greater than N; characterized in that - the first state parameter and the second state parameter are related to one another in such a way that each value that the second state parameter is able to assume can be used to definitively calculate one of the N discrete values for the first state parameter; a discrete definition of the first state parameter for determining non-functional properties of the technical system being taken as a basis for deriving a magnitude of the second state parameter; the magnitude having values in a larger number space than the first state parameter, or values in a continuous number space; the method also being characterized by the following steps: - obtaining a measurement result of at least one measurement on the system (1); - determining a current value for the second state parameter on the basis of the measurement result; the current value being determined directly or indirectly from the measurement result; - determining an assessment characteristic on the basis of the current value for the second state parameter and on the basis of a predetermined first critical value of the N discrete values for the first state parameter by evaluating an assessment function; the assessment function relating the second state parameter to the first critical value for the first state parameter.

2. Computer-implemented method according to Claim 1, the first critical value corresponding to a state of the system (1) in which the system (1) cannot meet a predetermined performance requirement and / or a predetermined reliability requirement and / or a predetermined safety requirement and / or a predetermined availability requirement.

3. Computer-implemented method according to either of the preceding claims, wherein - the first critical value is taken as a basis for determining a second critical value for the second state parameter; and - the assessment characteristic is determined on the basis of a distance of the current value for the second state parameter from the second critical value.

4. Computer-implemented method according to Claim 3, the first state parameter assuming the first critical value precisely when the second state parameter is less than or equal to the second critical value.

5. Computer-implemented method according to one of the preceding claims, the second state parameter quantifying a performance capability of the system (1), the performance capability of the system (1) being greater the greater the second state parameter.

6. Computer-implemented method according to one of the preceding claims, wherein the assessment characteristic is taken as a basis for making a prediction for the first state parameter.

7. Computer-implemented method according to Claim 6, wherein - an evolution of the assessment characteristic over time is determined by repeatedly performing the steps of obtaining the measurement result, determining the current value for the second state parameter and determining the assessment characteristic during an assessment period; and - the prediction for the first state parameter is made on the basis of the evolution of the assessment characteristic over time.

8. Computer-implemented method according to one of the preceding claims, wherein the assessment characteristic is taken as a basis for calculating a probability of a current value of the first state parameter corresponding to the first critical value.

9. Computer-implemented method according to one of the preceding claims, wherein the assessment characteristic is taken as a basis for calculating a probability of the first state parameter assuming the first critical value within a predetermined future period.

10. Computer-implemented method according to one of the preceding claims, wherein the assessment characteristic is taken as a basis for estimating a time at which the first state parameter is likely to assume the first critical value.

11. Method for assessing the state of a technical system (1), wherein a computer-implemented method according to one of the preceding claims is carried out and wherein the measurement result is generated by performing the at least one measurement on the system (1).

12. Method according to Claim 11, wherein the at least one measurement is performed by means of a measuring system (1) fully automatically according to a predetermined measurement protocol.

13. State assessment device (2) for assessing the state of a technical system (1) characterizable by a predetermined first state parameter and characterizable by a predetermined second state parameter, the state assessment device (2) being configured to carry out the method according to Claim 1.

14. State assessment device (2) according to Claim 13, comprising a measuring system (4) configured to perform the at least one measurement on the system (1).

15. Computer program product comprising commands that - when executed by a computer system, cause the computer system to carry out a computer-implemented method according to one of Claims 1 to 10; and / or - when executed by a state assessment device (2) according to Claim 14, cause the state assessment device (2) to carry out a method according to either of Claims 11 and 12.