PROCEDURES FOR TESTING CONTROL UNITS

Description

[0001] The invention relates to a method for testing control units, as well as a computing unit program and a computing unit program product. Furthermore, a test circuit and a corresponding test device are presented.

[0002] Control units are an important component in a wide variety of technical solutions. Their applications range from simple to essential tasks. Control units in vehicles, ships, and aircraft, in particular, are designed for essential tasks, meaning that safe operation cannot be guaranteed without their proper functioning.

[0003] Control units in vehicles, ships, and the aviation industry in general are developed according to the classic V-model. Requirements are formulated on the left-hand side (V), which are then tested or submitted for testing on the right-hand side (V). The V-model generally assumes individual, disjoint, unambiguous, atomic, and testable requirements. This is consistent with software requirements according to, for example, ISO 29148. During testing, the requirements are tested one at a time using various test design techniques. These techniques are described, for example, in ISO 26262 for safety-critical systems. Further, detailed procedures are described in ISO 29119.

[0004] Due to the sequential execution of the corresponding tests and thus the actual testing of the individual requirements, no statement can be made about any existing combinatorial coverage.

[0005] Previous methods and products output results in a simplistic form or highlight maximally redundant or omitted components without addressing any possible combinations. This prevents statements about combinatorial coverage, meaning that the functions related to the tested control unit are only tested individually and not in a higher-level context. This prevents statements about failed functions in their mutual dependence, which can sometimes lead to misinterpretation of a test quality measure. This can result in serious disadvantages both for the initial release of such a tested control unit and during corresponding operation of such a control unit.

[0006] In addition, the functions of control units are tested within their own systems. However, there are also increasingly scenarios in which the functions of individual control units may potentially influence each other. This circumstance is only partially, if at all, taken into account by existing test routines, which is therefore considered a disadvantage.

[0007] US 10,338,993 B1 concerns a computing unit that generates a test suite containing test cases for testing a system. A test condition in the test suite comprises one of different levels representing different options associated with a categorical factor for the system.

[0008] US 2020 / 0117587 A1 concerns a log file analysis in which acceptable deviations between log file entries are used to generate different patterns, whereby a comparison of log file entries with known acceptable success patterns is intended to provide an indication of system or application anomalies.

[0009] The invention is based on the object of providing an alternative method for testing control units which at least partially overcomes the disadvantages mentioned above.

[0010] The invention is described in the appended claims.

[0011] In a preferred embodiment of the invention, a computer-implemented method for testing control units is provided. Such a method comprises the following steps: First, at least two control units are provided and operated, each control unit comprising a plurality of functions, including at least one function dependent on at least one condition and at least one invariant function. A test routine is then provided and operated which tests the function of each of the operated control units. The functions to be tested are then divided into at least two groups within the respective control units, a first group comprising functions which are classified as safety-relevant functions by the user, and all further groups comprising the remaining functions. The respective functions of the at least two groups are then tested.It is then determined to what extent a tested function per control unit has a connection to at least one tested function from at least one other group in the same control unit.

[0012] This is followed by combining the tested functions of the first group with the connected and tested functions from all other groups within the respective control unit by forming cross products between the respective functions. Initial intermediate results of each combination are then generated.

[0013] Subsequently, by forming cross products between the respective functions, it is determined to what extent a tested function of the first group from a first control unit has a connection to at least one tested function from a second control unit. Second intermediate results are then generated for respective combinations between the at least two control units. The first intermediate results and second intermediate results are then combined to form a combination test result. Finally, this combination test result is then provided.

[0014] In this way, it is possible to provide an alternative method for testing control units that at least partially overcomes the disadvantages mentioned above.

[0015] The presented method makes it possible to quickly and accurately obtain insights into cross-ECU relationships with regard to the respective functions of individual ECUs, so that these insights can be used for a corresponding quality process. It is thus possible to provide insights that can then be used to specifically validate multiple ECUs. The presented method for testing the ECU landscape is particularly advantageous for more complex products with a correspondingly large number of ECUs that sometimes influence each other.

[0016] The presented method is therefore fundamentally designed to ensure coverage of safety functions, for example, while taking functional features into account. Especially when the control unit to be tested is intended for a product that has safety-relevant features in some areas, it is important to determine whether all safety functions are effective, also with regard to their respective interdependencies. By testing safety-relevant functions and related additional functions equally individually and then linking them to provide a higher-level statement in the form of a combined result, it is possible to achieve an expanded safety quality measure.For example, if a standalone function that is not considered safety-relevant in itself is of some importance for a safety-relevant function, the combined result provides initial indications of the extent of such problematic cross-connections. The combined result can thus be used in subsequent steps, for example, for improving the control unit, and provides experts with initial concrete indications of potential weak points.

[0017] These described advantages can now also be achieved with two or more control units using the method presented, whereby the mutual influence of the control units is taken into account.

[0018] In a further preferred embodiment of the invention, a computing unit program is provided, which comprises program code means for performing all steps according to one of claims 1 to 6 when the program is executed on a computing unit. The aforementioned advantages, to the extent transferable, also apply to the provided computing unit program. The computing unit on which the computing unit program is provided can, for example, be any computer, in particular a computer designed for use in a vehicle.

[0019] In a further preferred embodiment of the invention, a computing unit program product is provided, which comprises program code means stored on a computer-readable medium for carrying out the method according to one of claims 1 to 6 when the program is running in a computing unit. The aforementioned advantages also apply, to the extent transferable, to the presented computing unit program product.

[0020] The computing unit for which the computing unit program product is intended can, for example, be any computer, in particular a computer designed for use in a vehicle. It can also mean, for example, a computer suitable for use in the aircraft industry, biotechnology, or shipping.

[0021] In a further preferred embodiment of the invention, it is provided that a test circuit is provided which is configured to carry out a method according to one of claims 1 to 6. The aforementioned advantages also apply, to the extent transferable, to the test circuit presented.

[0022] In a further preferred embodiment of the invention, it is provided that a test device is provided which comprises a test circuit according to claim 9. The aforementioned advantages also apply, to the extent transferable, to the test device presented.

[0023] Further preferred embodiments of the invention result from the remaining features mentioned in the subclaims.

[0024] According to the invention, a first group comprises functions that are user-defined and classified as safety-relevant functions, while all further groups comprise the remaining functions. Thus, respective partial statements regarding respective functions can be initially determined in order to subsequently perform targeted combinatorics. User-defined settings can be implemented, so that the presented method can be advantageously adapted to specific application scenarios in simple steps, resulting in a particularly flexible method.

[0025] The invention also provides for the combination to be performed by forming the respective cross product. Precise combinatorics thus increases the significance of the final result, allowing for a targeted interpretation.

[0026] Furthermore, in another preferred embodiment of the invention, the preceding steps are performed in conjunction with at least one piece of software of the control unit to be tested. The method is therefore flexible in its use and can be designed for this scenario.

[0027] In a further preferred embodiment of the invention, the preceding steps are performed in conjunction with at least one hardware component of the control unit to be tested. The method is therefore flexible in its use and can be designed for this scenario.

[0028] Finally, in a further preferred embodiment of the invention, it is provided that respective time information of respective functions to be tested is determined and wherein it is determined to what extent a respective tested function per control unit has a connection to at least one tested function of the same control unit from at least one other group in general and with regard to respective time information and wherein these tested functions are combined with respectively connected and tested functions from all other groups per control unit taking into account a general reference and respective time information and wherein respective third intermediate results are created from these respective combinations and wherein these respective third intermediate results replace the first intermediate results.

[0029] By evaluating the temporal relationships between functional and safety-relevant functions, it is now possible to make statements about the initiation / failure of safety features over time. It is thus possible, for example, to obtain clues in the combination result and, at least in part, already in the intermediate results, which indicate when, in relation to temporal relationships, which conditions of safety features are violated. These findings can then be used for appropriate product optimization. It is also possible to obtain insights into the extent to which temporal relationships of the functions being tested result in security gaps that then need to be closed. In other words, temporal information is used in this way when forming cross-products across multiple control units.This makes it possible to create cross-product relationships across multiple control units with respect to their respective temporal information, so that these connections are also taken into account in the test routine to be performed. This allows for a cross-product analysis across multiple control units, taking into account connections related to temporal relationships between the control units, thus achieving a particularly meaningful combination result.

[0030] The presented method and the presented objects can be advantageously employed and used within the automotive industry, for example. In particular, the objects can be used in connection with any vehicle. For example, to test respective control units in vehicles, in particular passenger cars. These can be partially or fully autonomous vehicles. Uses of the disclosed objects and the method are also conceivable in connection with aviation safety and other safety standards. Furthermore, use is conceivable wherever combined product testing (electronics / software) is relevant. In addition to the automotive industry, use is also conceivable in aircraft construction, in avionics in general, or in shipbuilding. Use is also conceivable in the medical industry.Ultimately, it is conceivable to use it in conjunction with any SW / HW (electronic) testable products that are equipped with the appropriate debug interfaces and / or bus interfaces and / or signal interfaces.

[0031] The invention is explained in more detail below using an exemplary embodiment and the accompanying drawings. The figures show: Figure 1 shows a schematic representation of a flowchart of a method for testing control units; Figure 2 shows a schematic representation of a computing unit program; Figure 3 shows a schematic representation of a computing unit program product; Figure 4 shows a schematic representation of a test circuit; Figure 5 shows a schematic representation of a test device.

[0032] Figure 1shows a schematic representation of a flowchart 100 of a method for testing control units. In a first method step 110, at least two control units are provided and operated, wherein each control unit comprises at least one function dependent on at least one condition and at least one invariant function. In a second method step 120, a test routine is provided and operated which tests the respective operated control units with regard to their function. In a third method step 130, the functions to be tested are divided into at least two groups within the respective control units. In a fourth method step 140, the respective functions of the at least two groups are tested. For example, it is conceivable that these respective test results are created in the form of log files.In a fifth method step 150, it is determined to what extent a tested function for each control unit has a connection to at least one tested function from at least one other group in the same control unit. The log files created in the previous step are available for this purpose, for example, so that this information can be extracted accordingly. In a sixth method step 160, the tested functions are combined with respectively connected and tested functions from all other groups within the respective control unit. In a seventh method step 170, respective first intermediate results of respective combinations are created. In an eighth method step 180, it is determined to what extent a tested function from a first control unit has a connection to at least one tested function from a second control unit.In a ninth method step 190, respective second intermediate results of respective combinations between the at least two control units are generated. In a tenth method step 200, the first intermediate results and second intermediate results are combined to form a combination test result. In an eleventh method step 210, the combination result is provided.

[0033] Testing of the respective functions can be carried out using suitable test coverage metrics based on ISO29119-4. In this way, it is possible to determine a certain level of quality. It is also conceivable that the assignment of the function to respective groups is carried out in such a way that a differentiation of the functions based on their properties into at least one test observer and invariants is possible. A test observer is a monitor of a function. It is based on a requirement on the left-hand side of the V-model. It is thus the actual test algorithm for testing the specific requirement on which it is based. In contrast to an invariant, a test observer is linked to at least one condition.For example, if the residual voltage is greater than or equal to six volts and the product is in the switched-on state, with this state also declared as "active mode," then the support power should always be greater than or equal to a specified value. In contrast to a test observer, an invariant is a condition that is valid at any given time. For example, it can be specified that a product must respond on the bus within a specified period of time, such as five milliseconds. Thus, an invariant is a special observer, although this invariant is specified without conditions.

[0034] Based on the general statement that a certain function stored in a control unit and assigned to a specific group needs to be tested, it is conceivable that this testing could be carried out as follows: If the test observers or invariants are defined as coverage points, this results in respective coverage sets, which are primarily traversed linearly in requirements-based test routines. The respective coverage points, sorted into a set, are based on the requirements being tested and thus also on the respective characteristics stored in the requirements and their respective definitions, which are each associated with the functions to be tested. For example, a set of coverage points results as follows: G = Ci mit i = 0 … n − 1

[0035] In requirements-based testing, the individual Ci are all run when the requirements are fully implemented in the test. This makes it possible to generate a basis for content-related requirement coverage that is representative of the testing of a respective stand-alone function. Before performing the actual test, for example, certain Ci can be set to zero. Each Ci is incremented as soon as it is run in the test. A structure in the Ci can be used to note whether the test for that Ci has passed or failed. At the end of the requirements-based test runs, no Ci may be empty. It is therefore possible to create or output statistics on the evaluation of G, which note which Ci has passed or failed and how often.

[0036] Based on these test runs and the resulting information, an extension to the power of the combinatorics of Ci is planned. This combinatorics is particularly important for establishing appropriate feature cross-product coverage, for example, in safety-relevant products. Especially in safety-relevant products, it is important to determine whether all safety functions are effective under various functional aspects. This is possible using the respective coverage point sets from the various groups.

[0037] S = { CSi} with i = 0 ... k - 1, where k is the number of safety functions.

[0038] The remaining functions must be put into another set, or group, accordingly: F = Ci mit i = 0 … n − k − 1 .

[0039] By forming the cross product of the sets S and F, it is then possible to cover the safety functions taking into account the functional characteristics: K = S × F .

[0040] Here, too, it is possible to determine an aggregated combined result beyond the intermediate results, which can then be made available for further analysis. In other words, the number of successful and unsuccessful tests is displayed for each Ki after a respective process. A complete run of all Ki can then be performed, for example, in combination with a corresponding robustness test procedure.

[0041] Using the presented method, it is now possible to perform feature coverage analyses using observers and invariants for multi-ECU validation.

[0042] The presented method thus advantageously complements the development of a quality metric that focuses on multi-ECU validation. The presented method aims to establish a reference to multiple ECUs during testing in order to establish a reference to the validation of multiple ECUs and thus functions of a complex product, such as a vehicle, in particular a passenger car, an aircraft, or similar.

[0043] Complex technical products have functions that can span multiple control units, which are arranged at least partially networked within the product. An example of this is a parking assist system in a vehicle. When validating this function, the dimension of the K-matrix increases by the reference to the respective control unit: K = S t , Ej × F t , Ej

[0044] This means that both safety functions S from control unit E index j with time reference t and functional functions F with time reference t and control unit E index j.

[0045] In other words, for example, the function F of the control unit E index j represents a subfunction of the overarching function such as the parking steering assistant, i.e. for example an overarching function G.

[0046] It then follows that the subfunctions F of E index j are then cross-producted with the respective cross products of their associated safety functions S of E index j. This allows us to determine whether each subfunction F has been fully tested with its safety functions S. This results in a statement that each subfunction F will operate its safety mechanisms S under malfunctions.

[0047] This present form K is accordingly helpful in order to now also build a corresponding metric across control units in order to combine safety features of Ej with functional features as well as safety features of the other control units Ej - 1.

[0048] The following example from the area of ​​vehicle use is conceivable: Raising and lowering a side window in the vehicle must not affect the steering system being switched off. This example therefore concerns both the window control unit and the steering control unit. Both control units are connected to the vehicle's associated bus, in which these two control units are located, as well as to the vehicle's electrical system. Bus messages on the bus for the window control unit must not affect the steering control unit. Likewise, there must be no undesired influence on the steering control unit due to raising / lowering the window and the resulting voltage fluctuations on the vehicle's electrical system.

[0049] This example is relatively simple. A possible solution is disjoint protection, i.e., without the metrics used here, i.e., each control unit is independent (and thus without the possibility of mutual influence).

[0050] However, there are more complex scenarios. For example, there are vehicle assistance systems that, due to the functions they must perform, cannot be operated independently. For example, in a park steering assistance system, the cooperation of several control units is absolutely necessary to enable the desired complex function of the parking process or general driver support. The presented metric is therefore applicable to both simple and more complex cases, whereby the temporal reference can also be considered optional. In other words, in a basic variant of the presented method, it is also conceivable to focus only on the mutual influences across multiple control units.

[0051] The evaluation of the tuples in K provides the test analyst in the overall functional validation, for example at the overall vehicle level, with metrics to make statements about the quality of the requirement combinatorial tests.

[0052] Safety functions, such as sudden braking, must not occur in conjunction with the interior light being switched on. Conversely, it would certainly be interesting to obtain a statement from the metric that a safety feature was not triggered, even though the expectation was that it should have occurred.

[0053] The metrics obtained using the presented method also allow the evaluation of functions and safety functions across ECUs in their temporal context. The presented method can also be applied to mechanical tests, for example, if one wants to describe the search smoke of mechanical movements.

[0054] It is also conceivable that the coverage points could be structured into more complex scenarios. For example, one might want to see the test sequence from C1 → C4 → C5. A coverage feature path analysis can be used to describe statements about the stability of the product under certain functional conditions.

[0055] In a software-based test, the coverage points can be easily measured by tapping the software variables, structures, and information. In an electronics-hardware-based test, the information can be obtained from the data streams of the corresponding MCU debug interfaces. This can be done, for example, in combination with a real-time analysis method designed for functional testing of ECU hardware and software.

[0056] Because data collection is independent of the test location and test environment, it is universal. Clear measurement criteria for ECU software quality allow products to be brought to market more reliably. Measures can be monitored, and any improvements to the ECU software can therefore be advantageously measured using the presented method.

[0057] The presented method is particularly useful for vehicle construction. Due to the steadily growing proportion of software functions in vehicles, their awareness among customers is also increasing. Customers are increasingly understanding that the corresponding software of control units determines the quality of their vehicle. With the presented method, it is possible to test even more complex relationships with manageable effort and, in particular, to test networked structures under the conditions mentioned during the development phase and then improve them depending on the results. The presented method is suitable for establishing a quality metric or using the respective results obtained for this purpose. In particular, a combination of hardware- and software-based tests is also possible in order to thus establish an overall quality metric.

[0058] For increasingly complex control units, including those used in autonomous vehicles, metrics are needed that provide information about product quality. The method presented here demonstrates one way to achieve this.

[0059] Through clear metrics as well as their evaluation and use, a user group is able to make product decisions based on data in order to achieve a qualitative improvement before market launch if necessary.

[0060] Figure 2 shows a schematic representation of a computing unit program 10. Such a computing unit program 10 comprises program code means 12 for carrying out all steps according to one of claims 1 to 4 when the program is executed on a computing unit not shown in detail.

[0061] Figure 3shows a schematic representation of a computing unit program product 14. Such a computing unit program product 14 comprises program code means 12 which are stored on a computer-readable medium in order to carry out the method according to one of claims 1 to 4 when the program is running in a computing unit.

[0062] Figure 4 shows a schematic representation of a test circuit 16. Such a test circuit 16 is configured to carry out a method according to one of claims 1 to 4.

[0063] Figure 5 shows a schematic representation of a test device 18. This test device 18 comprises a test circuit 16 according to claim 7. Reference symbol

[0064] 10Arithmetic unit program 12Program code means 14Arithmetic unit program product 16Test circuit 18Test device 100Flowchart 110First process step 120Second process step 130Third process step 140Fourth process step 150Fifth process step 160Sixth process step 170Seventh process step 180Eighth process step 190Ninth process step 200Tenth process step 210Eleventh process step

Claims

1. A computer-implemented method for testing control devices, comprising the steps of: • Providing and operating (110) at least two control devices, wherein each control device comprises multiple functions, including at least one function dependent on at least one condition and at least one invariant function; • Providing and operating (120) a test routine which tests the respective operated control devices with regard to their function; • Dividing (130) the functions to be tested into at least two groups within the respective control devices, wherein a first group comprises functions which by user definition are classified as safety-relevant functions and wherein all further groups comprise the remaining functions; • Testing (140) each of the functions of the at least two groups; • Determining (150) to what extent a tested function per control device has a connection to at least one tested function from at least one other group in the same control device; • Combining (160) tested functions of the first group with respectively connected and tested functions from all other groups within the respective control device by forming cross products between the respective functions; • Creating (170) respective first intermediate results of each of the combinations; • Determining (180), by forming cross products between the respective functions, the extent to which a tested function of the first group from a first control device has a connection to at least one tested function from a second control device; • Creating (190) respective second intermediate results of respective combinations between the at least two control devices; • Consolidating (200) the first intermediate results and second intermediate results into a combined test result; • Providing (210) the combined result.

2. The method according to claim 1, wherein the previous steps are carried out in connection with at least one software of the control device to be tested in each case.

3. The method according to any of the preceding claims 1 and 2, wherein the previous steps are carried out in connection with at least one hardware component of the control device to be tested in each case.

4. The method according to any of the preceding claims, wherein respective time information of respective functions to be tested is determined and wherein it is determined to what extent each tested function per control device has a connection to at least one tested function of the same control device from at least one other group in general and with respect to respective time information and wherein these tested functions are combined with respectively connected and tested functions from all other groups per control device taking into account a general reference and respective time information by forming cross products and wherein respective third intermediate results are created from these respective combinations and wherein these respective third intermediate results replace the first intermediate results.

5. A computing unit program (10) comprising program code means (12) for performing all the steps according to any of claims 1 to 4 when the program is executed on a computing unit.

6. The computing unit program product (14) comprising program code means (12) which are stored on a computer-readable medium for performing the method according to any of claims 1 to 4 when the program runs in a computing unit.

7. A test circuit (16) which is designed to execute a method according to any of claims 1 to 4.

8. A test apparatus (18) comprising a test circuit (16) according to claim 7.