Test priority evaluation method based on airborne embedded software

Abstract

The invention discloses a test priority evaluation method based on airborne embedded software, which relates to the field of software engineering verification, and comprises the following steps: decomposing functional modules corresponding to the airborne embedded software based on requirements to obtain a plurality of decomposed functional modules; obtaining a safety influence degree grade score and an importance grade score of the decomposition function module; calculating a comprehensive grade score according to the safety influence degree grade score and the importance grade score; defining priorities and permissions thereof according to the comprehensive grade score; and formulating a test priority result judgment criterion according to the priority permission definition so as to obtain a test priority result matrix according to the criterion. According to the method, a high-safety evaluation criterion for safety influence degree analysis in a function failure mode and the importance and judgment criterion of the function module are defined, and a comprehensive evaluation criterion is defined by combining the maturity and the importance of the software module and a project development stage; the practical application requirement of a project is met, and the test work can be safely, reliably and quickly deployed and developed.

Description

Technical Field

[0001] The present invention relates to the field of software engineering verification, and in particular to a test priority evaluation method based on airborne embedded software. Background Art

[0002] Software testing, as a form of software verification, plays a crucial role in the software engineering process. In GJB5000B, software verification and validation are a separate process area with practical requirements. In DO-178C, software verification objectives account for 50%, demonstrating the importance of software verification.

[0003] As the pace of software engineering project development cycles continues to accelerate and the strictness of software product delivery quality continues to increase, in different software development stages and within the limited software development cycle given, which functional module tests should be prioritized to ensure that the software delivered at this stage meets customer delivery requirements, avoids major or above quality problems and major safety hazards, and causes customer complaints. This is an urgent problem that has been faced and must be solved.

[0004] Most existing methods focus on requirements prioritization or test case execution priority algorithms. These methods aim to quickly and accurately develop requirements or efficiently and effectively execute test cases when batching test cases for regression testing. These methods are primarily applicable to non-airborne and non-embedded software domains. There is a lack of methods tailored to airborne embedded software, and there is a lack of targeted research on how to quickly and reliably assess requirements based on their criticality, safety, and other characteristics and select the appropriate requirements-based tests for each development phase. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a test priority evaluation method based on airborne embedded software, which solves the problems of the prior art priority evaluation means being complex, costly and inefficient.

[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a test priority evaluation method based on airborne embedded software, comprising: Decompose the functional modules corresponding to the onboard embedded software based on the requirements to obtain several decomposed functional modules; Obtain the safety impact level score and importance level score of the decomposed functional module based on the failure mode and the defined safety impact level judgment criteria and functional module importance judgment criteria; Determine whether each decomposed functional module is a reusable functional module, and adjust the security impact level score and importance level score of each functional module according to the defined functional module reuse criteria; The comprehensive grade score is calculated based on the security impact grade score and importance grade score of each functional module; if the decomposed functional module is a non-reusable functional module, the comprehensive grade score of the decomposed functional module is calculated directly; if the decomposed functional module is a reusable functional module, the comprehensive grade score of the decomposed functional module is calculated after updating the security impact grade score and importance grade score of the reusable functional module according to the functional module reuse criteria; Define priorities and authorities based on comprehensive grade scores; formulate test priority result evaluation criteria based on priority authority definitions and obtain a test priority result matrix based on the criteria.

[0007] The beneficial effects of the present invention are: 1. Through the formulation of various criteria, testers can be assisted in quickly formulating reasonable test work plans, quickly determining the scope of testing, prioritizing, and implementing testing work in a purposeful and directional manner, thus avoiding the loss of manpower costs, time costs, and quality costs caused by conducting comprehensive testing within a limited time.

[0008] 2. Since the required functional modules and software security analysis results have been identified and the reuse attributes have been marked in the development phase of the software project development process, when entering the test verification phase, the test priority result matrix will be quickly obtained based on this and the evaluation criteria in the evaluation method, and the required functions to be tested will be determined, so that the testing work can be carried out quickly.

[0009] 3. The evaluation process of the present invention is clear and simple, the evaluation criteria are quantitative, clear and standardized, the algorithm is simple and easy to understand, and the method is easy to operate, which is convenient for technical personnel to understand and use, reduces the complexity of test priority sorting, saves labor costs, and improves work efficiency.

[0010] 4. The evaluation method developed in the present invention is based on multi-dimensional considerations such as failure safety analysis, key software features, and reuse maturity. It is safe, reliable, and low-risk. It can ensure that when the software development cycle is short and the delivery task is heavy, the testing work that needs to be carried out in the current development stage can be determined intuitively and quickly, ensuring that the quality of the delivered software does not have quality problems and safety hazards of the important level or above, and meets the customer delivery requirements of the stage. As the stages progress and the corresponding testing work is carried out, the quality problems and safety hazards of the delivered software will also be eliminated. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A flowchart of a test priority evaluation method based on airborne embedded software is provided in an embodiment. DETAILED DESCRIPTION

[0012] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0013] like Figure 1 As shown, in one embodiment of the present invention, taking the software requirement functions of an air data computer as an example, a test priority evaluation method based on airborne embedded software includes the following steps: S1. Decompose the functional modules corresponding to the airborne embedded software based on the requirements to obtain several decomposed functional modules.

[0014] By communicating with customers, users, or other stakeholders, we gather requirements for the onboard embedded software and then categorize and organize them. Based on the results of the requirements analysis, we further decompose the core functional modules into smaller functional modules, thus obtaining several decomposed functional modules. Each decomposed functional module is designed to be simple and independent, reducing coupling between modules and facilitating subsequent testing and understanding of software functionality by developers.

[0015] In this embodiment, the decomposed functional modules include a CPU initialization module, an external interface initialization module, an internal interface initialization module, a parameter initialization module, a sensor characteristic data reading module, a correction data reading module, a CPU instruction checking module, a memory checking module and a power detection module.

[0016] S2. Obtain the security impact level score and importance level score of the decomposed functional module.

[0017] The specific steps include: S21. Identify the failure mode of each decomposed functional module, that is, identify the failures or abnormal conditions that may occur in each functional module during operation.

[0018] S22. Define the criteria for judging the safety impact level of functional modules; As shown in Table 1, the safety impact level includes four levels: high, medium, low, and slight. This quantifies the safety impact level, facilitates intuitive understanding, and facilitates subsequent assessments. Failure modes with a high safety impact level include: causing the product to malfunction or freeze, hindering the implementation of key product functions, resulting in the loss of key product functions, causing fatal errors to pilots, and causing fatal consequences to the application of interactive systems. Failure modes with a medium safety impact level include: adversely affecting key functions with no workaround solution, hindering the implementation of key functions, resulting in the loss of key functions, adversely affecting key functions with known workaround solutions, causing serious errors to pilots, causing serious consequences to the application of interactive systems, and adversely affecting key functions with no workaround solution. Failure modes with a low safety impact level include: hindering the implementation of general functions or causing the loss of general functions; adversely affecting important functions but with known workarounds; causing minor consequences for pilot operation errors; causing minor consequences for interactive system applications; affecting user satisfaction but not affecting the operation of the product software or the implementation of software functions; adversely affecting general functions for which no workaround is available. Failure modes with a minor safety impact level include: adversely affecting general functions but with known workarounds; causing inconvenience to technicians but not hindering the completion of their work; not meeting user requirements, hindering maintenance or auxiliary testing, not affecting user requirements; and other impacts that do not affect user requirements.

[0019] Table 1

[0020] Among them, "fatal" means causing aircraft destruction or casualties, "serious" means increasing the crew workload, causing physical discomfort, pain or injury to the crew members; "minor" means a slight increase in the crew workload, causing slight physical discomfort to the crew members.

[0021] S23. Perform a safety impact analysis on the failure mode corresponding to each decomposed functional module according to the functional module safety impact level judgment criteria and obtain a safety impact level score A.

[0022] In this embodiment, when the CPU initialization module fails to initialize, the CPU does not work properly, resulting in the software not running or running abnormally, and the product cannot run normally. The security impact level is "high" and the corresponding score is 6 points; when the external interface (bus, analog quantity, discrete quantity) fails to initialize, the software input and output communication is abnormal, the product cannot run normally, and the security impact level is "high" and the corresponding score is 6 points; when the internal interface (AD, FD, interrupt) fails to initialize, the software input and data acquisition are abnormal, the product cannot run normally, and the security impact level is "high" and the corresponding score is 6 points; when the parameter initialization is incomplete, random numbers may participate in the program logic, resulting in function implementation errors, and the security impact level is "medium" and the corresponding score is 5 points; when the sensor characteristics When data reading fails / errors occur, the atmospheric parameter solution data is abnormal / inaccurate, the main functions of the aircraft are lost, and the safety impact level is "high", with a corresponding score of 6 points; when correction data reading fails / errors occur, the uncorrected atmospheric data calculation data may be inaccurate, may not meet the user's data accuracy requirements / flight bus requirements, and have an adverse impact on key functions, with a safety impact level of "medium", with a corresponding score of 5 points; when the CPU instruction check fails / errors, if the CPU instruction check does not pass, it means that there may be a problem with the internal logic device of the CPU, which may affect the correctness of the data solution, or the data is abnormal. If the CPU instruction check is implemented incorrectly, it is impossible to correctly judge whether the CPU instruction is correct, resulting in false alarms or no fault reporting, which is difficult to troubleshoot and locate, affecting user satisfaction. Because it is a fault reporting mechanism, there are no task handling measures for actual faults. At the same time, if a fault is not reported, fault feedback can be obtained through other channels. Therefore, incorrect implementation of this function does not affect the implementation of the main function. The safety impact level is "low" and the corresponding score is 3 points. When the memory check fails / errors, if the memory check does not pass, it means that there is a problem with the memory access area, and data cannot be accessed normally. This may affect the correct application of program variables and cause program abnormal operation. If the memory check is implemented incorrectly, the fault will be falsely reported or not reported, which will increase the difficulty of troubleshooting and affect user satisfaction. It does not affect the implementation of the main function. The safety impact level is "low" and the corresponding score is 3 points. When the power supply test fails / errors, if the power supply test does not pass, the abnormal power supply will cause the corresponding hardware to operate abnormally, causing abnormal software operation. If the power supply test is implemented incorrectly, it will cause the fault to be falsely reported or not reported, which will increase the difficulty of troubleshooting and affect user satisfaction. It does not affect the implementation of the main function. The safety impact level is "low" and the corresponding score is 3 points.

[0023] S24. Define the criteria for judging the importance of functional modules; As shown in Table 2, the importance levels include three levels: critical functions, important functions, and general functions. Among them, the basic configuration functions that ensure the operation of the product software and the main functional requirements proposed by users are critical functions; when the product software needs to run in the air, other functional requirements proposed by users except critical functions are important functions; software functions that do not affect flight safety and need to be implemented on the ground are general functions.

[0024] Table 2

[0025] Among them, the basic configuration function refers to the initialization function that enables the software to run on the target machine; the backbone functional requirements refer to the product input, output, and logic solution / control functions implemented by the software to meet user needs.

[0026] S25. Determine the importance level score B of each decomposed functional module according to the functional module importance judgment criterion; In this example, the CPU instruction check module, memory check module, and power supply detection module are all assigned an "Important Function" importance rating of 2. All other modules are assigned a "Critical Function" importance rating of 3. This quantifies the importance of these modules, facilitating intuitive understanding and subsequent evaluation. This example does not address modules corresponding to general functions, which primarily refer to software functions that require ground execution and do not impact flight safety, such as ground maintenance functions.

[0027] S26, determining whether each decomposed functional module is a reusable functional module; S27. Define the reuse criteria for functional modules; As shown in Table 3, the corresponding relationship between the security impact level (score) of reusable functional modules and non-reusable functional modules and the importance level (score) of functional modules is shown. The security impact level and importance level of reusable functional modules are degraded.

[0028] Table 3

[0029] If the decomposed functional module is a non-reusable functional module, the comprehensive grade score of the decomposed functional module is directly calculated; If the decomposed functional module is a reusable functional module, the safety impact level score and importance level score of the reusable functional module shall be determined as A1 and B1 respectively according to the functional module reuse criteria.

[0030] The modules involved in this embodiment are all non-reusable modules. If they are reusable modules, it is only necessary to update the security impact level score and the function importance level score according to the reuse criteria, and then calculate the comprehensive level score to perform the final evaluation.

[0031] S3. Calculate the comprehensive score based on the security impact score and importance score; The calculation expression of the comprehensive grade score is: Comprehensive grade score = security impact grade score × importance grade score.

[0032] In this embodiment, the comprehensive grade scores of the CPU initialization module, external interface initialization module, internal interface initialization module and sensor characteristic data reading module are all 18, the comprehensive grade scores of the parameter initialization module and correction data reading module are 15, and the rest are 6 points.

[0033] S4. Define priorities and permissions based on the comprehensive rating score; As shown in Table 4, the priorities are divided into six domains according to the comprehensive grade scores. Priorities are defined for each domain and corresponding permissions are assigned to each priority to support the final test priority result evaluation.

[0034] Table 4

[0035] S5. Develop test priority result evaluation criteria based on the priority authority definition combined with the development stage and software level.

[0036] As shown in Table 5, the development phase includes ground joint testing, first flight, and delivery flight. Software levels are categorized as general software, important software, and critical software. By considering the three dimensions of comprehensive score, development phase, and software importance, we developed a test priority evaluation criterion to comprehensively evaluate and generate a test priority matrix for the corresponding required functional modules.

[0037] Table 5

[0038] S6. Based on the test priority result evaluation criteria, a comprehensive assessment is conducted to provide a test priority result matrix corresponding to each required functional module, which directly guides the test work arrangement.

[0039] As shown in Table 6, according to the test priority result evaluation criteria, the test priority result matrix of all functional modules is obtained after comprehensive evaluation.

[0040] Table 6

[0041] Finally, software testing work is carried out according to the priority result matrix.

[0042] The present invention is an evaluation method developed based on multi-dimensional considerations such as failure safety analysis, key software features, and reuse maturity. It is safe, reliable, and low-risk. It can ensure that when the software development cycle is short and the delivery task is heavy, the testing work that needs to be carried out in the development stage can be determined intuitively and quickly, ensuring that the quality of the delivered software does not have quality problems and safety hazards of important level or above, and meets the customer delivery requirements of the stage. As the stages progress and the corresponding testing work is carried out, the quality problems and safety hazards of the delivered software will also be eliminated.

Claims

1. A test priority evaluation method based on airborne embedded software, characterized in that: include: Decompose the functional modules corresponding to the onboard embedded software based on the requirements to obtain several decomposed functional modules; Obtain the safety impact level score and importance level score of the decomposed functional module based on the failure mode and the defined safety impact level judgment criteria and functional module importance judgment criteria; Determine whether each decomposed functional module is a reusable functional module, and adjust the security impact level score and importance level score of each functional module according to the defined functional module reuse criteria; The comprehensive grade score is calculated based on the security impact grade score and importance grade score of each functional module; if the decomposed functional module is a non-reusable functional module, the comprehensive grade score of the decomposed functional module is calculated directly; if the decomposed functional module is a reusable functional module, the comprehensive grade score of the decomposed functional module is calculated after updating the security impact grade score and importance grade score of the reusable functional module according to the functional module reuse criteria; Define priorities and permissions based on comprehensive rating scores; According to the definition of priority authority, the test priority result evaluation criteria are formulated and the test priority result matrix is ​​obtained according to the criteria.

2. The method according to claim 1, characterized in that The security impact level scores and importance level scores of the decomposed functional modules are as follows: Identify the failure modes of each decomposed functional module; Define the criteria for judging the safety impact level of functional modules; According to the functional module safety impact level judgment criteria, the safety impact analysis of the failure mode corresponding to each decomposed functional module is performed and the safety impact level score A is obtained; Define the criteria for judging the importance of functional modules; Determine the importance level score B of each decomposed functional module according to the functional module importance judgment criteria.

3. The method according to claim 2, characterized in that The importance levels include three levels: critical functions, important functions and general functions. Among them, the basic configuration functions that ensure the operation of product software and the main functional requirements proposed by users are critical functions; the functions that product software needs to run in the air and other functional requirements proposed by users except critical functions are important functions; software functions that need to be implemented on the ground without affecting flight safety are general functions.

4. The method according to claim 3, characterized in that The safety impact level includes four levels: high, medium, low and slight. Among them, failure modes with a high safety impact level include: causing the product to not work properly or crash, hindering the realization of the product's key functions, causing the loss of the product's key functions, causing erroneous instructions to the pilot's operation, which is fatal, and causing fatal consequences to the application of the interactive system.

5. The method according to claim 4, characterized in that Failure modes with a medium safety impact level include: adverse effects on critical functions for which there is no workaround solution, hindering the implementation of important functions, resulting in loss of important functions, adverse effects on critical functions for which workaround solutions are known, the consequences of causing erroneous instructions to pilots are serious, the consequences for the application of interactive systems are serious, adverse effects on important functions for which there is no workaround solution.

6. The method according to claim 4, characterized in that Failure modes with a low safety impact level include: hindering the realization of general functions or causing the loss of general functions, adversely affecting important functions, but with known workarounds, the result of causing erroneous instructions to pilots is minor, the consequences for the application of interactive systems are minor, affecting user satisfaction but not affecting the operation of the product software and the realization of software functions, adversely affecting general functions, and without any workarounds.

7. The method according to claim 4, characterized in that Failure modes with a minor safety impact level include: adverse effects on general functions, but workarounds are known; inconvenience to technical personnel, but does not hinder the completion of the work; non-user needs, not conducive to self-maintenance or auxiliary detection, no impact on user needs; other impacts that do not affect user needs.

8. The method according to claim 2, characterized in that The calculation expression of the comprehensive grade score is: Comprehensive grade score = security impact grade score × importance grade score.

9. The method according to claim 2, characterized in that In the functional module reuse criteria, degradation scores are used for the security impact level and importance level of reused functional modules.

10. The method according to claim 1, characterized in that In the test priority result evaluation criteria, the comprehensive grade scores are divided into 6 grade domains, priorities are defined for each grade domain, and corresponding permissions are assigned to each priority, including the development stage and software grade standards applicable to functional module testing.

Images