METHOD AND DEVICE FOR EVALUATING THE QUALITY OF EXECUTION AFTER TRANSLATION OF SOFTWARE

Abstract

METHOD AND DEVICE FOR EVALUATING THE QUALITY OF EXECUTION AFTER TRANSLATION OF SOFTWARE The present invention relates to a method (100) for evaluating the quality of execution after translation of software by a target processor, comprising translating (110) a source code (CS) of the software into a main machine code by means of a main translation device and into an alternative machine code by means of an alternative translation device, the main machine code and the alternative machine code being executable by the target processor, the main translation device and the alternative translation device being dissimilar; for at least one predetermined test case (CT), executing (120) the main machine code and the alternative machine code by means of the target processor, to respectively provide a main execution result and an alternative execution result;and providing (130) an indicator (IN) of the quality of execution of the software by the target processor based on the result of a comparison of the main execution result and the alternative execution result. Figure for abstract: 1;

Description

Title of the invention: METHOD AND DEVICE FOR EVALUATING THE QUALITY OF EXECUTION AFTER TRANSLATION OF SOFTWARE

[0001] The present invention relates to a method for evaluating the quality of execution after translation of software by a target processor, in particular of software used on an on-board computer for aeronautical applications requiring a high level of security.

[0002] In the fields of aeronautics and autonomous vehicles, many aircraft or vehicle control functions are provided by on-board computers.

[0003] For critical functions, for example for the flight controls of an aircraft or the braking of a car, high operational reliability is required.

[0004] Operational safety can be characterized by attributes including reliability, maintainability, availability, safety-harmlessness and safety-confidentiality.

[0005] To ensure or evaluate the operational safety of a critical computer, four categories of means are implemented: fault avoidance through a rigorous development process, fault elimination during validation phases, fault prediction, and fault tolerance.

[0006] Fault tolerance techniques are mainly based on the principles of monitoring and redundancy.

[0007] Asymmetric redundancy consists of using hardware and / or software elements that come from two different development processes from the same functional specifications.

[0008] In particular, N-self-testing programming is based on the execution of N software programs in parallel and the implementation of an algorithm for comparing the execution results.

[0009] In the simplest case where N=2, a computer dedicated to a given system function is composed of two different hardware computing units, preferably physically segregated, i.e. placed in separate boxes, for example side by side.

[0010] The first unit is a COM control unit. The COM control unit develops the control instructions and transmits them to the relevant actuating devices as well as to the second unit, called the MON monitor unit which monitors the COM control unit, and where appropriate to other computers ensuring the same function.

[0011] The MON monitor unit produces the same instructions as the COM control unit but only monitors COM.

[0012] The results calculated by the two COM and MON units are compared. If a deviation that exceeds a tolerance threshold authorized by the system designers is observed, the computer loses the system function concerned as well as its associated MON, and if possible, passes control to another computer to process the lost function.

[0013] Ideally, COM and MON units rely on different hardware, software developed by different teams and in different languages.

[0014] This property is the dissimilarity of hardware, software and processes and can also be called dissimilarity.

[0015] However, in certain fields and in particular the field of aeronautics, it may be difficult or even impossible to multiply operating systems, so that common mode defects may remain.

[0016] Furthermore, in the areas of critical functions, it is not currently envisaged to take advantage of the options for optimizing translation devices, due to an increase in the level of risk that these options may entail, in particular due to the presence of design or programming defects (in English "bugs") in these options.

[0017] An aim of the invention is then to propose a method making it possible to evaluate the quality of the execution of a given software by a target processor after translation, and in particular after translation by means of an optimized translation device, this method making it possible to reinforce operational safety when totally asymmetrical redundancy is not possible.

[0018] To this end, the invention relates to a method for evaluating the quality of execution after translation of software by a target processor, comprising the steps of: (a) translating a source code of the software into a main machine code by means of a main translation device and into an alternative machine code by means of an alternative translation device, the main machine code and the alternative machine code being executable by the target processor, the main translation device and the alternative translation device being dissimilar; b) for at least one predetermined test case, executing the main machine code and the alternative machine code by means of the target processor, to provide respectively a main execution result and an alternative execution result; and (c) providing an indicator of the quality of execution of the software by the target processor based on the result of a comparison of the main execution result and the alternative execution result.

[0019] The implementation of an alternative translation device dissimilar to the main translation device makes it possible in particular to highlight certain common mode defects induced by the source code translation device into machine code itself, when a single translation device is used in two redundant units such as a COM command unit and a MON monitor unit.

[0020] To ensure operational safety when a translation device is common to two redundant units, one choice may be to minimize optimizations of the latter. This choice in fact makes it possible to have good knowledge of the defects of the translation device and, consequently, to consider that these defects will be observable during well-chosen unit or integration tests.

[0021] However, the lack of optimization on the translation device results in the machine code resulting from the translation not being optimal. In order to compensate for the suboptimality of the machine code to be executed, the use of a more powerful target processor may be necessary.

[0022] In the field of aeronautics, choosing a translation device that is not or only slightly optimized is therefore not always technically feasible.

[0023] In the case where an optimized translation device is required, the invention therefore consists, counter-intuitively and contrary to this prerequisite, in generating an alternative machine code from an alternative translation device dissimilar to the optimized translation device.

[0024] The alternative translation device may be suboptimal, or even non-optimized, or even optimized according to different criteria. Whatever the case, the dissimilarity between the translation devices makes it possible to increase the level of confidence in the result of execution of the software by the target processor via the machine code generated by the optimized translation device. In the case where the translation device is optimized, the method according to the invention makes it possible to benefit from the advantages of optimization, for example in terms of reducing the consumption of the target processor, while guaranteeing the level of security required for the specific case.

[0025] In particular, if the results of execution of the software by the target processor via the main machine code and via the alternative machine code are identical, the probability that defects related to the optimized compiler are present is low. On the contrary, if these results are different, a defect related to the optimized compiler can be suspected and sought.

[0026] According to other advantageous aspects of the invention, the method comprises one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0027] - the main translation device and the alternative translation device are compilers, the execution step then being subsequent to the translation step;

[0028] - the main translation device and the alternative translation device are interpreters, the translation and execution stages being in this case simultaneous;

[0029] - the method comprises a preliminary step of optimizing the translation device principal according to at least one optimization criterion;

[0030] - the at least one optimization criterion is chosen from a translation time of the source code and / or execution of the machine code on the target processor, a memory cost of the machine code and / or energy consumption of the target processor;

[0031] - the preliminary step i) of optimization comprises the optimization of the device of alternative translation, the at least one optimization criterion of the alternative translation device being different from the at least one optimization criterion of the main translation device;

[0032] - the method comprises at least one prior step of testing the source code comprising at least one unit test and / or at least one integration test and / or at least one functional test using at least one prior test case;

[0033] - the at least one predetermined test case of the execution step comprises at least a preliminary test case;

[0034] - the indicator of the quality of the execution of the software by the target processor is binary.

[0035] The invention also relates to a device for evaluating the quality of the execution of software by a target processor comprising the means for implementing the method according to any one of the preceding embodiments.

[0036] The invention will appear more clearly on reading the description which follows, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0037] [Fig-1] [Fig.l] represents an embodiment of the device according to the invention;

[0038] [Fig.2] [Fig.2] represents an embodiment of the method according to the invention in form of flowchart, the method being implemented by the device of [Fig.l].

[0039] The device 5 for evaluating the quality of execution after translation of software by a target processor according to the invention is described with reference to [Fig.l].

[0040] The software is for example configured to allow the implementation of a function, in particular a critical function, in an aircraft or a land vehicle (not shown), for example an autonomous land vehicle, when it is implemented on a processor of this aircraft or this vehicle, during an operating phase of the software.

[0041] The processor for the software exploitation phase is the target processor.

[0042] The software is developed prior to the implementation of the method according to the invention on the device 5, in the form of a source code CS.

[0043] The CS source code comprises a set of instructions written in a programming language. given programming, in a human-readable form.

[0044] The programming language chosen is not important for the invention. As illustrative examples, the programming language in which the CS source code is written is chosen from Python, C, ADA, or RUST.

[0045] The device 5 comprises a memory 10, a main translation processor 15A, an alternative translation processor 15B and an execution processor 20. The memory 10 is configured to store the source code CS and to exchange data with the main translation processor 15A, the alternative translation processor 15B and the execution processor 20.

[0046] The main translation processor 15A may be the target processor itself or another processor.

[0047] The main translation processor 15A implements a main translation device, configured to produce a main machine code of the software from the source code CS.

[0048] The main machine code comprises a set of instructions directly executable by the target processor and possibly by one or more other processors embedded in the aircraft or ground vehicle. In other words, the main machine code is an executable binary file.

[0049] The main translation device is therefore chosen not only according to the programming language in which the CS source code is written, but preferably also according to the specifications of the target processor.

[0050] Advantageously, the main translation device is optimized according to at least one optimization criterion.

[0051] This optimization criterion can in particular be chosen from: - a criterion relating to the execution time of the main machine code by the target processor, - a criterion relating to a memory footprint during the execution of the main machine code by the target processor, - a criterion relating to a memory footprint for storing the main machine code in a memory of the target processor, - and / or a criterion relating to the energy consumption of the target processor during the execution of the main machine code by the target processor.

[0052] In a first embodiment, the main translation device is a compiler.

[0053] In this case, the main machine code is generated by the main translation device prior to a step of executing this main machine code.

[0054] The main machine code can then be stored in memory 10 and then executed later.

[0055] If the operating system of the target processor is the Unix system, the compiler is for example chosen from the compilers of the GNU compiler collection (in English, “GNU Compiler Collection” GCC).

[0056] In the case where the main translation device is a compiler, an object code can be generated intermediately in an intermediate language such as RTL (Register Transfer Language). The object code is an intermediate between the CS source code and the machine code.

[0057] Alternatively, the primary translation device is an interpreter. In this case, the machine code is generated source code line by source code line, in response to a software execution command.

[0058] The machine code is then executed as it goes without prior storage in the memory 10 of all the translated CS source code instructions.

[0059] This is for example the case if the programming language of the CS source code is the Python language.

[0060] Alternatively, the translation device is a just-in-time compiler that combines a "classic" compiler, as described above, and an interpreter.

[0061] The alternative translation processor 15B may be the target processor, the main translation processor 15A, or another processor.

[0062] The alternative translation processor 15B implements an alternative translation device, configured to produce an alternative machine code of the software from the source code CS of this software.

[0063] Like the main machine code, the alternative machine code comprises a set of instructions directly executable by the target processor.

[0064] The alternative translation device is otherwise dissimilar from the primary translation device.

[0065] By the qualifier "dissimilar", or equivalently "dissimilar", it is meant that the development of the alternative translation device and that of the main translation device involved at least one development specification, and / or at least one design step, and / or at least one technology, the notion of technology encompassing the programming language, and / or at least one human being for the development of at least a fraction of code which differs between the two translation devices.

[0066] For example, the alternative translation device and the primary translation device may have been developed by two different development teams. For example, the primary translation device may be from the GCC collection and the alternative translation device may be from the LLVM (Low Level Virtual Machine) compiler collection.

[0067] Alternatively or in addition, only one of the main translation device and the translation device may be optimized.

[0068] Alternatively or additionally, the main translation device and the translation device may be optimized according to at least one different optimization criterion.

[0069] In the same way as for the main translation device, the alternative translation device can be of the compiler, interpreter or on-the-fly compiler type.

[0070] In a particular embodiment, the main translation device and the alternative translation device are of the same type, this type being chosen from a compiler, an interpreter or an on-the-fly compiler.

[0071] The execution processor 20 is configured to receive the main machine code, the alternative machine code and at least one predetermined CT test case stored in the memory 10.

[0072] The processor 20 is configured to provide a main execution result (respectively an alternative execution result) at the end of the execution of the main machine code (respectively of the alternative machine code) for the at least one predetermined CT test case.

[0073] The execution processor 20 is either the processor on which the software will be implemented for the software exploitation phase, or a processor which has the same characteristics as the processor used in the software exploitation phase, for example a processor corresponding to the same manufacturer reference.

[0074] The device 5 also comprises a comparison processor 30, configured to receive the main execution result and the alternative execution result, compare these two results and provide an indicator IN of the quality of the execution of the software by the target processor as a function of the result of the comparison.

[0075] The comparison processor 30 may be the execution processor 20, the main translation processor 15A or the alternative translation processor 15B, or another processor.

[0076] The method 100 for evaluating the quality of the execution of software by a target processor by means of the device 5 is described with reference to [Fig.2].

[0077] The method 100 comprises a step 110 of translating a source code CS of the software.

[0078] The translation step 110 comprises a main translation sub-step 110A for providing the main machine code from the source code CS.

[0079] The main translation substep 110A is implemented by means of the main translation processor 15A.

[0080] The translation step 110 comprises an alternative translation sub-step 110B to provide alternative machine code from CS source code.

[0081] The alternative translation substep 110B is implemented by means of the alternative translation processor 15B.

[0082] The alternative translation substep 110B and the main translation substep 110A can be performed in parallel or successively.

[0083] Due to the dissimilarity of the primary and alternate translation devices, the primary machine code and the alternate machine code have a reduced probability of exhibiting the same translation-related defects.

[0084] The more numerous the causes of dissimilarity between the main and alternative translation devices, the lower this probability.

[0085] The method 100 also comprises a step 120 of executing the main machine code and the alternative machine code by means of the execution processor 20, to respectively provide a main execution result and an alternative execution result from at least one predetermined CT test case.

[0086] The execution step 120 comprises a main execution sub-step 120A in which the execution processor 20 receives said test case CT and the main machine code.

[0087] The execution processor 20 then executes the main machine code generated in the translation sub-step 110A for this test case CT.

[0088] The CT test case may be a known test case, i.e. one from a prior IT development process. For example, one or more CT test cases may be from a prior software development process for one or more software programs, for which the DAL-A criticality level of the DO-178C standard applicable on January 1, 2012 was required.

[0089] Alternatively or in addition, one or more CT test cases may be developed ad hoc for the method 100. For example, a CT test case may be designed specifically to activate an optimization function of the translation device that one wishes to use.

[0090] Each CT test case includes a test function.

[0091] The test function may in particular be chosen to enable a unit test to be carried out, i.e. to control the execution of a predetermined portion of the CS source code.

[0092] The test function can also be chosen to enable an integration test to be carried out, i.e. to check the correct execution of a combination of at least two predetermined portions of the CS source code by checking any intermediate states of the target processor.

[0093] The test function can also be chosen to enable a functional test to be carried out, i.e. to check the correctness of the result provided at execution. of the source code.

[0094] At the end of the main execution sub-step 120A, a main execution result is provided by the execution processor 20.

[0095] The particular nature of the main execution result depends on said CT test case and is not important for the method.

[0096] The main execution result comprises, for example, at least one binary value and / or at least one integer or real value and / or at least one character string.

[0097] The main execution result may comprise a sub-execution result for each of the individual test cases of said CT test case.

[0098] In a particular embodiment, the execution result is a byte whose elements are of the same type or of different types.

[0099] The execution step 120 also comprises an alternative execution sub-step 120B in which the execution processor 20 receives said test case CT and the alternative machine code.

[0100] The execution processor 20 then executes the alternative machine code generated in the translation substep 110B for this test case CT.

[0101] At the end of the alternative execution sub-step 120B, an alternative execution result is provided by the execution processor 20.

[0102] The test case CT being the same for the main execution sub-steps 120A and alternative 120B, the preceding explanations for the main execution result are valid mutatis mutandis for the alternative execution result.

[0103] The alternative execution substep 120B and the main execution substep 120A can be performed in parallel or successively.

[0104] In the case where the main (respectively alternative) translation device is an interpreter, the main execution sub-step 120A (respectively alternative 120B) is simultaneous with the main translation sub-step 110A (respectively alternative 110B).

[0105] In the case where the main (respectively alternative) translation device is a compiler, the main execution sub-step 120A (respectively alternative 120B) can be carried out immediately after the main translation sub-step 110A (respectively alternative 110B), or alternatively in a deferred manner.

[0106] At the end of the execution step 120, the method 100 comprises a step 130 of providing the indicator IN of the quality of the execution of the software by the comparison processor 30.

[0107] The providing step 130 comprises receiving the main execution result and the alternative execution result and then comparing these two results by the comparison processor 30.

[0108] The comparison of the main execution result and the alternative execution result may comprise the two-by-two comparison of all the elements of the main result and the execution result if these results comprise several elements.

[0109] At the end of the comparison of the main execution result and the alternative execution result, the comparison processor 30 provides as output the indicator IN of the quality of the execution of the software by the target processor.

[0110] The IN indicator can be binary.

[0111] For example, if the alternative execution result and the main execution result are not equal in the computational sense, i.e. taking into account coding rounding, the IN indicator can take the value 0 or FALSE. Otherwise, the IN indicator can take the value 1 or TRUE.

[0112] The IN indicator thus makes it possible to obtain information on the impact of the main translation device on the execution of the software by the target processor.

[0113] Since the primary and alternative translation devices are dissimilar, if an error is introduced due to the primary translation device, this error will manifest itself with a high probability through an observable difference between the alternative execution result and the primary execution result, provided that the CT test case is properly chosen.

[0114] In particular, if the main translation device is optimized, at least one CT test case can be designed in relation to one or more optimization functions used.

[0115] If no difference is observed, the level of confidence in the execution of the software by the target processor, or in other words the operational safety, is therefore reinforced thanks to the method 100.

[0116] The IN indicator can also be composite. In particular, it can comprise a set of sub-indicators IN_i, for example binary, a sub-indicator IN_i being associated with each of a plurality of test cases CT.

[0117] Alternatively or in addition, the indicator IN may comprise a probability P reflecting the probability that the observed difference is linked to the main or alternative translation device.

[0118] For example, if a plurality of N CT test cases are implemented, each associated with a binary sub-indicator IN_i equal to zero in the absence of an observable difference between the alternative execution result and the main execution result and equal to 1 otherwise, the probability P may be an average, possibly weighted, of the sub-indicators IN_i obtained for the N CT test cases.

[0119] Whatever the nature of the IN indicator, at the end of the supply step 130, it is therefore possible to decide on the disqualification or the start of the exploitation of the main machine code if the method 100 is implemented during a validation phase prior to the exploitation of the software.

[0120] It is also possible to decide whether or not to maintain the exploitation of the main machine code if the method 100 is implemented during the exploitation phase of the software.

[0121] If a difference between the alternative execution result and the main execution result is observed via the IN indicator, this does not necessarily mean that this difference is related to the main translation device. The difference may in particular also be a manifestation of an error related to the alternative translation device or even an error related to the source code CS or related to the execution processor 20 itself.

[0122] Advantageously, the method 100 comprises a prior step of testing the source code CS comprising at least one unit test and / or at least one integration test and / or at least one functional test by means of at least one prior test case.

[0123] Advantageously, at least one prior test case is included in the at least one predetermined test case CT used in the execution step 120.

[0124] Alternatively or in addition, the method 100 may comprise an optional step 140 of analysis of all the elements involved from the source code CS of the software to the execution of the software by the execution processor 20.

[0125] For the analysis step 140, a processor, for example the comparison processor 30, receives the indicator IN and provides a priority verification instruction, either of the source code CS, or of the target processor, or of verification and / or replacement of the main translation device.

[0126] In particular, each test case CT can be designed to test one or more particular elements involved from the source code CS of the software up to the execution of the software by the execution processor 20. In the case where the indicator IN is composite and comprises a set of sub-indicators IN_i, the analysis of the different indicators IN_i for which a difference between the alternative execution result and the main execution result is observed can make it possible to automatically develop the priority verification instruction.

Claims

Claims

1. Method (100) for evaluating the quality of execution after translation of software by a target processor, comprising the steps of: a) translating (110) a source code (CS) of the software into a main machine code by means of a main translation device and into an alternative machine code by means of an alternative translation device, the main machine code and the alternative machine code being executable by the target processor, the main translation device and the alternative translation device being dissimilar; b) for at least one predetermined test case (CT), executing (120) the main machine code and the alternative machine code by means of the target processor, to provide respectively a main execution result and an alternative execution result;and c) providing (130) an indicator (IN) of the quality of the execution of the software by the target processor based on the result of a comparison of the main execution result and the alternative execution result.;

2. The method (100) of claim 1, wherein the primary translation device and the alternative translation device are compilers, the execution step then being subsequent to the translation step.

3. The method (100) of claim 1, wherein the primary translation device and the alternative translation device are interpreters, the translation and execution steps in this case being simultaneous.

4. Method (100) according to any one of the preceding claims, comprising a prior step of: i) optimizing the main translation device according to at least one optimization criterion.

5. Method (100) according to claim 4, wherein the at least one optimization criterion is chosen from a translation time of the source code and / or execution of the machine code on the target processor, a memory cost of the machine code and / or an energy consumption of the target processor.

6. A method (100) according to claim 4 or claim 5 wherein the prior step i) of optimizing comprises optimizing the alternative translation device, the at least one optimization criterion of the alternative translation device being different from at least one optimization criterion of the primary translation device.

7. Method according to any one of the preceding claims, comprising at least one prior step of: ii) testing the source code (CS) comprising at least one unit test and / or at least one integration test and / or at least one functional test by means of at least one prior test case.

8. The method of claim 7, wherein the at least one predetermined test case (CT) of the executing step (120) comprises at least one prior test case.

9. Method according to any one of the preceding claims, in which the indicator (IN) of the quality of the execution of the software by the target processor is binary.

10. Device (5) for evaluating the quality of the execution of software by a target processor comprising the means for implementing the method according to any one of the preceding claims.

Images