A method and system for testing the robustness of an airborne real-time operating system kernel interface

Abstract

This invention provides a robustness testing method and system for the kernel interface of an airborne real-time operating system. It employs a multi-dimensional equivalence class partitioning model for parameter classification, systematically enumerating six categories—legal values, boundary values, illegal values, null pointers, extreme values, and permission exceptions—based on two dimensions: data type and semantic role. This avoids the problem of overlooking exception categories in manual, experience-based design. Orthogonal lists are used for parameter combination, ensuring that any equivalence class combination between any two parameters is tested at least once. At the test execution level, four mechanisms are deployed: watchdog deadlock detection, memory sentinel out-of-bounds detection, kernel panic callback capture, and non-volatile progress marker restart detection, making result determination independent of the API return value itself. The structured test report generated by this invention includes coverage statistics, detailed abnormal test cases, and defect location information, serving as objective evidence of DO-178C robustness testing activities and meeting the airworthiness certification requirements for test sufficiency and traceability.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to a method and system for robust testing of the kernel interface of an airborne real-time operating system. Background Technology

[0002] In avionics systems, the Real-Time Operating System (RTOS), as the runtime platform for airborne software, directly impacts the safety of the entire avionics system through the stability and robustness of its kernel. DO-178C (Software Considerations in the Certification of Airborne Systems and Equipment) explicitly requires that executable object code be robust to abnormal inputs and conditions. This means that RTOS kernel system calls (APIs) must be able to safely intercept and return appropriate error codes when receiving non-target parameters, without causing system crashes, deadlocks, or data corruption.

[0003] The RTOS kernel API (Application Programming Interface) has a rich variety of parameter types and complex semantics. However, in current engineering practices, robustness testing of the RTOS kernel interface primarily relies on the following methods:

[0004] The first type involves manually writing exception test cases. Test engineers write test cases with exception parameters for each API based on their experience. The problem with this approach is that the workload is disproportionate to the parameter space: a typical RTOS kernel contains dozens to hundreds of system calls, each with multiple parameters, and each parameter has multiple types of exception values, making it almost impossible to exhaustively combine them manually. More importantly, manual design easily overlooks the interaction effects between parameters—some defects are only exposed under specific parameter combinations, and exceptions are often not found by testing the exception values ​​of each parameter individually. In addition, different testers have different experience and styles, and the coverage of the same interface varies in different projects.

[0005] The second type is random fuzzing, which injects random values ​​into API parameters. While this method can indeed uncover some robustness flaws, the distribution of random values ​​is uncontrolled—many test cases may repeatedly fall into the same region of the parameter space, resulting in insufficient coverage of critical exception categories such as boundary values ​​and null pointer exceptions. Furthermore, the results of random testing are difficult to reproduce and trace, which falls short of the DO-178C requirement for test traceability.

[0006] Since both of the above methods have their shortcomings—manual methods are incomplete and inefficient, while random methods lack systematicity and traceability—practical engineering requires a robust testing scheme that can systematically classify API parameters, use mathematical methods to ensure coverage of abnormal combinations, and automatically execute and determine the results. Summary of the Invention

[0007] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a method and system for testing the robustness of the kernel interface of an airborne real-time operating system, in order to solve the technical problems existing in the prior art.

[0008] To achieve the above and other related objectives, this invention provides a method for robust testing of the kernel interface of an airborne real-time operating system, comprising the following steps:

[0009] The parameter equivalence class partitioning includes: parsing the interface definition of the RTOS kernel API, extracting all parameters and their data types for each API function, and establishing an equivalence class partitioning model for each parameter according to its data type and semantic role;

[0010] Orthogonal defect combination includes: selecting an orthogonal table based on the number of parameters of each API function and the number of equivalence classes of each parameter, and combining the representative values ​​of the equivalence classes of each parameter according to the column mapping relationship of the orthogonal table to generate a robust test case set for each API; wherein each test case contains at least one specific set of values ​​for all parameters of the corresponding API function;

[0011] Automated test execution includes: sequentially calling the corresponding RTOS kernel API for each test case and collecting kernel state snapshots through a pre-deployed or real-time automated test execution framework;

[0012] Kernel status monitoring and anomaly detection include: real-time monitoring of the RTOS kernel's running status, detection of abnormal events, and recording, when an abnormal event is detected, the test case parameter combination that triggered the corresponding abnormal event, the anomaly type, and a kernel status snapshot; wherein, the abnormal event includes at least one of deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart;

[0013] Test result determination and report generation include: automatically determining the status of each test case based on API return values ​​and kernel status monitoring results, and generating a structured robustness test report based on the status of all test cases. The robustness test report includes test coverage statistics, summary of abnormal test cases, anomaly attribution analysis, and defect location information; wherein, the status of the test case is either pass or fail.

[0014] Optionally, the process of establishing the equivalence class partitioning model includes:

[0015] For pointer type parameters, the established equivalence classes include: valid pointer pointing to the target memory region, null pointer, dangling pointer pointing to a freed memory region, unaligned pointer, and pointer to a read-only memory region;

[0016] For integer type length or size parameters, the established equivalence classes include: normal values ​​within the target range, zero values, negative values, upper bound values ​​of the data type, and outlier values ​​outside the target range;

[0017] For the timeout parameter, the established equivalence classes include: normal timeout value, zero timeout, infinite wait flag value, negative timeout value, and extreme timeout value close to the upper bound of the data type.

[0018] For permission flags or option parameters, the established equivalence classes include: target flag value or flag combination, all-zero flag, all-one flag, non-target flag value, and mutually exclusive flag;

[0019] For enumeration type parameters, the established equivalence classes include: all target enumeration items, integer values ​​outside the enumeration range, and values ​​with the upper bound of the enumeration type plus one.

[0020] Optionally, the process of combining orthogonal defects further includes:

[0021] The number of parameters k and the number of equivalence classes for each parameter in the target API function are statistically analyzed to determine the number of factors and the number of levels for each factor in the orthogonal array.

[0022] Select the orthogonal table that satisfies the following conditions from the pre-built mixed-level orthogonal table library: k factors, the number of levels of each factor is not less than the number of equivalence classes of the corresponding parameters, and the smallest number of rows. When there is no exactly matching orthogonal table, select the most approximate orthogonal table and perform horizontal merging or pruning adjustments.

[0023] Each column of the orthogonal array is mapped to a parameter, and each level value in the column is mapped to an equivalence class representative value of the parameter. Each row of the orthogonal array is decoded into a complete test case, which contains the specific values ​​of all parameters.

[0024] After performing orthogonal defect combination in the orthogonal array, add benchmark test cases where all parameters take preset values ​​and extreme test cases where all parameters take extreme outliers simultaneously to supplement the peripheral scenarios not covered by orthogonal defect combination.

[0025] Optionally, the kernel status monitoring and anomaly detection process further includes:

[0026] Deadlock detection includes: starting a watchdog timer before each API call; if the API call does not return within the set maximum allowed time, it is judged as a suspected deadlock, and the status of all tasks, the resources they hold, and the resources they are waiting for are recorded.

[0027] Memory out-of-bounds detection includes: performing integrity checks on the preset memory sentinel region before and after API calls; if the characteristic value of the preset memory sentinel region changes, it is determined to be a memory out-of-bounds write, and the corrupted memory address range is recorded.

[0028] Kernel panic detection includes: registering a kernel panic callback hook; when the RTOS kernel triggers the error handling process, the kernel panic callback hook captures error context information, including exception type, error code, fault instruction address, and call stack backtracking.

[0029] Abnormal restart detection includes: maintaining test progress markers in a non-volatile storage area, and checking whether the test progress markers match the expected next test case number at each startup. If they do not match, it is determined that the previous API call caused an unexpected system restart.

[0030] Optionally, the process of determining test results and generating reports further includes:

[0031] For test cases called with non-target parameters, if the API returns a predefined error code and the kernel status is normal, the test is considered to have passed.

[0032] For test cases called with non-target parameters, if the API does not return an error code but executes normal logic, or causes at least one of the following abnormal events to be detected: deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart, the test is considered to have failed.

[0033] For benchmark test cases called with target parameters, if the API returns the expected result and the kernel status is normal, the test is considered to have passed; if the API returns an exception or the kernel status is abnormal, the test is considered to have failed.

[0034] Optionally, the anomaly attribution analysis includes employing a minimum anomaly parameter combination identification strategy based on binary search, associating each failed test case with its parameter equivalence class combination to identify the minimum parameter combination causing the anomaly, thereby locating the cause of the defect; the specific steps are as follows:

[0035] Step A1: For a test case that is determined to have failed, record its complete parameter combination as P = {p1,p2, ..., pk}, where k is the total number of parameters of the API and pi is the current value of the i-th parameter;

[0036] Step A2: Identify the parameters in the parameter set P that take illegal equivalence class values, and denote them as the abnormal parameter set S = {ps1, ps2, ..., psm}, where m is the number of parameters that take illegal equivalence class values, and m ≤ k; if m = 1, then the single abnormal parameter is the smallest parameter combination that causes the abnormality, and the attribution analysis ends.

[0037] Step A3: If m ≥ 2, divide the abnormal parameter set S into two equal parts. and the second half ;in, Indicates rounding down;

[0038] Step A4: Construct a verification test case V_front, where the parameters in S_front retain their original illegal values, while all parameters in S_back are replaced with the representative values ​​of the legal equivalence classes of the corresponding parameters, and the remaining parameters that originally have legal values ​​remain unchanged; execute the verification test case V_front and observe whether the same type of abnormal event is still triggered;

[0039] Step A5: Construct a verification test case V_back, where the parameters in S_back retain their original illegal values, while all parameters in S_front are replaced with the representative values ​​of the legal equivalence classes of the corresponding parameters, and the remaining parameters that originally have legal values ​​remain unchanged; execute the verification test case V_back and observe whether the same type of abnormal event is still triggered;

[0040] Step A6: Based on the execution results of steps A4 and A5, recursively reduce according to the following rules:

[0041] If V_front triggers an exception but V_back does not, then the exception parameter set is reduced to S_front, and steps A3 to A6 are repeated for S_front to perform recursive binary search.

[0042] If V_back triggers an exception but V_front does not, then the set of exception parameters is reduced to S_back, and steps A3 to A6 are repeated for S_back to perform recursive binary search.

[0043] If both V_front and V_back trigger exceptions, it means that S_front and S_back each independently contain parameters that can trigger exceptions. Recursively perform binary search on S_front and S_back independently, and finally take the set with the fewest elements among all independent recursive results as the minimum exception parameter combination.

[0044] If neither V_front nor V_back triggers an exception, it means the exception is triggered by the interaction of parameters in S_front and S_back, and a combination of both is required to reproduce the exception. In this case, S_front and S_back are further reduced by binary search, and unnecessary parameters are gradually removed using cross-validation: each exception parameter in S is replaced with a valid value and the test is re-executed. If the exception disappears after replacing a parameter, then that parameter is a necessary parameter for triggering the exception and is retained; if the exception still exists after replacement, then that parameter is unnecessary and is excluded. After traversing all exception parameters, the remaining set of necessary parameters is the minimum combination of parameters that causes the exception.

[0045] Step A7: When the set of abnormal parameters is reduced to the point where it cannot be reduced further (i.e., the abnormality disappears after each parameter in the set is replaced with a valid value), output the current set as the final minimum combination of abnormal parameters, and record this combination, along with the abnormality type and fault scene information, in the defect location information of the test report.

[0046] Optionally, the RTOS kernel API includes at least one of the following: management API, semaphore API, message queue API, timer API, and memory management API.

[0047] This invention also provides a robustness testing system for the kernel interface of an airborne real-time operating system, the system comprising:

[0048] The parameter equivalence class partitioning module is used to parse the interface definition of the RTOS kernel API, extract all parameters and their data types for each API function, and establish an equivalence class partitioning model for each parameter according to its data type and semantic role.

[0049] The orthogonal defect combination module is used to select an orthogonal table based on the number of parameters of each API function and the number of equivalence classes of each parameter, and combine the representative values ​​of the equivalence classes of each parameter according to the column mapping relationship of the orthogonal table to generate a robust test case set for each API; wherein, each test case contains at least one specific set of values ​​for all parameters of the corresponding API function;

[0050] The automated execution and monitoring module is used to sequentially call the corresponding RTOS kernel API for each test case through a pre-deployed or real-time automated test execution framework, and to collect kernel state snapshots; and to monitor the running status of the RTOS kernel in real time, detect whether abnormal events occur, and when abnormal events are detected, record the test case parameter combination, exception type, and kernel state snapshot that triggered the corresponding abnormal event; wherein the abnormal events include at least one of deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart; and to automatically determine the status of each test case based on the API return value and kernel state monitoring results, and generate a structured robustness test report based on the status of all test cases, wherein the robustness test report includes test coverage statistics, abnormal test case summary, abnormal attribution analysis, and defect location information; wherein the status of the test cases is either pass or fail.

[0051] Optionally, the orthogonal defect combination module further includes the following process for performing orthogonal defect combination:

[0052] The number of parameters k and the number of equivalence classes for each parameter in the target API function are statistically analyzed to determine the number of factors and the number of levels for each factor in the orthogonal array.

[0053] Select the orthogonal table that satisfies the following conditions from the pre-built mixed-level orthogonal table library: k factors, the number of levels of each factor is not less than the number of equivalence classes of the corresponding parameters, and the smallest number of rows. When there is no exactly matching orthogonal table, select the most approximate orthogonal table and perform horizontal merging or pruning adjustments.

[0054] Each column of the orthogonal array is mapped to a parameter, and each level value in the column is mapped to an equivalence class representative value of the parameter. Each row of the orthogonal array is decoded into a complete test case, which contains the specific values ​​of all parameters.

[0055] After performing orthogonal defect combination in the orthogonal array, add benchmark test cases where all parameters take preset values ​​and extreme test cases where all parameters take extreme outliers simultaneously to supplement the peripheral scenarios not covered by orthogonal defect combination.

[0056] Optionally, the process of kernel status monitoring and anomaly detection performed by the automated execution and monitoring module further includes:

[0057] Deadlock detection includes: starting a watchdog timer before each API call; if the API call does not return within the set maximum allowed time, it is judged as a suspected deadlock, and the status of all tasks, the resources they hold, and the resources they are waiting for are recorded.

[0058] Memory out-of-bounds detection includes: performing integrity checks on the preset memory sentinel region before and after API calls; if the characteristic value of the preset memory sentinel region changes, it is determined to be a memory out-of-bounds write, and the corrupted memory address range is recorded.

[0059] Kernel panic detection includes: registering a kernel panic callback hook; when the RTOS kernel triggers the error handling process, the kernel panic callback hook captures error context information, including exception type, error code, fault instruction address, and call stack backtracking.

[0060] Abnormal restart detection includes: maintaining test progress markers in non-volatile storage areas and checking whether the test progress markers are consistent with the expected next test case number at each startup. If they are inconsistent, it is determined that the previous API call caused an unexpected system restart.

[0061] The process of determining test results and generating reports by the automated execution and monitoring module also includes:

[0062] For test cases called with non-target parameters, if the API returns a predefined error code and the kernel status is normal, the test is considered to have passed.

[0063] For test cases called with non-target parameters, if the API does not return an error code but executes normal logic, or causes at least one of the following abnormal events to be detected: deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart, the test is considered to have failed.

[0064] For benchmark test cases called with target parameters, if the API returns the expected result and the kernel status is normal, the test is considered to have passed; if the API returns an exception or the kernel status is abnormal, the test is considered to have failed.

[0065] As described above, this invention provides a robustness testing method and system for the kernel interface of an airborne real-time operating system, which has the following beneficial effects: This invention adopts a multi-dimensional equivalence class partitioning model for parameter classification, systematically enumerating six categories—legal values, boundary values, illegal values, null pointers, extreme values, and permission exceptions—according to two dimensions: data type and semantic role. This avoids the problem of easily overlooking exception categories in manual, experience-based design. Furthermore, this invention utilizes orthogonal lists for parameter combination, mathematically ensuring that the equivalence class combination between any two parameters is tested at least once. Moreover, this invention simultaneously deploys four mechanisms at the test execution level: watchdog deadlock detection, memory sentinel out-of-bounds detection, kernel panic callback capture, and non-volatile progress marker restart detection, making the result determination independent of the API return value itself. Furthermore, the anomaly attribution analysis of this invention can identify the smallest parameter combination that triggers the anomaly, narrowing the scope of defect investigation from the entire set of inputs to specific values ​​of one or two parameters, shortening the developer's localization time. Furthermore, the structured test report generated by this invention includes coverage statistics, details of abnormal test cases, and defect location information, which can serve as objective evidence of DO-178C robustness testing activities, meeting the requirements of airworthiness certification for test sufficiency and traceability. Therefore, this invention requires no manual intervention throughout the entire process from parameter classification, test case generation, test execution to result judgment and report output. The same API set can be repeatedly regression tested, ensuring the consistency of test results across different batches. Therefore, this invention establishes multi-dimensional equivalence classes for each parameter of the RTOS kernel API according to data type and semantic role, and uses orthogonal lists to combine the representative values ​​of each parameter's equivalence classes, ensuring that every pairwise abnormal combination of parameters is covered with a smaller number of test cases. During the test execution phase, the kernel status is monitored through four mechanisms: watchdog timer, memory sentinel verification, kernel panic callback hook, and non-volatile progress marking. Deadlock, memory out-of-bounds access, kernel panic, and unexpected restarts are detected, and test results are automatically judged and test reports are generated. Therefore, this invention addresses the problems of low efficiency in manual code writing and lack of systematic random testing by proposing a robust testing scheme based on an orthogonal defect model. First, each API parameter is divided into equivalence classes according to parameter type and semantic role (covering legal values, boundary values, illegal values, null pointers, extreme timeouts, and permission flag anomalies). Then, an orthogonal array is used to combine the equivalence classes of each parameter. Mathematically, the orthogonal array guarantees that each pair of equivalence classes for each pair of parameters appears at least once, thus achieving pairwise interactive coverage with far fewer test cases than a complete Cartesian combination. Simultaneously, during the test execution phase, the API is automatically invoked on the target RTOS platform, the kernel status is monitored, and a test report is automatically generated after the results are determined. Attached Figure Description

[0066] Figure 1 A schematic diagram of the functional architecture of an airborne real-time operating system kernel interface robustness testing system provided in one embodiment;

[0067] Figure 2 A schematic diagram of the multidimensional equivalence class partitioning process for the parameter equivalence class partitioning module;

[0068] Figure 3 This is a schematic diagram of the orthogonal table mapping process for the orthogonal defect combination module;

[0069] Figure 4 A schematic diagram of the four-layer anomaly detection mechanism for the automated execution and monitoring module;

[0070] Figure 5 This is a diagram illustrating the complete testing process using the semTake() API as an example. Detailed Implementation

[0071] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0072] It should be noted that the illustrations provided in this embodiment are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0073] In an exemplary embodiment of the present invention, such as Figure 1 As shown, an airborne real-time operating system kernel interface robustness testing system is provided, including:

[0074] The parameter equivalence class partitioning module is used to parse the interface definitions of the RTOS kernel API, extract all parameters and their data types for each API function, and establish an equivalence class partitioning model for each parameter according to its data type and semantic role. The parameter types of the RTOS kernel API are rich and semantically complex. Taking a typical aerospace RTOS (such as VxWorks) as an example, an API function typically contains 3-8 parameters, covering various types such as pointers (e.g., task control block pointers, memory buffer pointers), integers (e.g., priority, buffer length, timeout), and enumerations (e.g., message priority flags, creation options). Each parameter type has multiple possible outliers: pointers may be NULL, dangling, or unaligned; length parameters may be zero, negative, or abnormally large; timeout parameters may be zero (immediate return), negative, or extremely large; and permission flags may be illegal combinations. The test space generated by these outlier combinations is enormous.

[0075] The orthogonal defect combination module is used to select an orthogonal table based on the number of parameters of each API function and the number of equivalence classes of each parameter, and combine the representative values ​​of the equivalence classes of each parameter according to the column mapping relationship of the orthogonal table to generate a robust test case set for each API; wherein, each test case contains at least one specific set of values ​​for all parameters of the corresponding API function;

[0076] The automated execution and monitoring module is used to sequentially call the corresponding RTOS kernel API for each test case through a pre-deployed or real-time automated test execution framework, and to collect kernel state snapshots. It also monitors the RTOS kernel's running status in real time, detects abnormal events, and, when an abnormal event is detected, records the test case parameter combination that triggered the event, the exception type, and the kernel state snapshot. Abnormal events include at least one of deadlock, memory out-of-bounds access, kernel panic, and unexpected system restarts. Furthermore, it automatically determines the status of each test case based on API return values ​​and kernel state monitoring results, and generates a structured robustness test report based on the status of all test cases. The robustness test report includes test coverage statistics, a summary of abnormal test cases, anomaly attribution analysis, and defect location information. The test case status is either pass or fail.

[0077] In some exemplary embodiments, the process of establishing the equivalence class partitioning model in the parameter equivalence class partitioning module may include:

[0078] For pointer type parameters, the established equivalence classes include: valid pointer pointing to the target memory region, null pointer (NULL), dangling pointer pointing to a freed memory region, unaligned pointer, and pointer to a read-only memory region.

[0079] For integer type length or size parameters, the established equivalence classes include: normal values ​​within the target range, zero values, negative values ​​(for signed types), upper bound values ​​of the data type, and outliers outside the target range. The target range can also be referred to as the legal range.

[0080] For the timeout parameter, the established equivalence classes include: normal timeout value, zero timeout (immediate return), infinite wait flag value, negative timeout value (for signed types), and extreme timeout value close to the upper bound of the data type.

[0081] For permission flags or option parameters, the established equivalence classes include: target flag value or flag combination, all-zero flag, all-one flag (all bits set), non-target flag value (value not belonging to the predefined flag set), and mutually exclusive flag. Among them, the target flag value can also be called the valid flag value, and the non-target flag value can also be called the invalid flag value.

[0082] For enumeration type parameters, the established equivalence classes include: all target enumeration items, integer values ​​outside the enumeration range, and values ​​one above the upper bound of the enumeration type. The target enumeration items can also be referred to as valid enumeration items.

[0083] In some exemplary embodiments, the process of orthogonal defect combination module performing orthogonal defect combination may include:

[0084] The number of parameters k and the number of equivalence classes for each parameter in the target API function are statistically analyzed to determine the number of factors and the number of levels for each factor in the orthogonal array; where k is a positive integer greater than or equal to 2.

[0085] Select the orthogonal table that satisfies the following conditions from the pre-built mixed-level orthogonal table library: k factors, the number of levels of each factor is not less than the number of equivalence classes of the corresponding parameters, and the smallest number of rows. When there is no exactly matching orthogonal table, select the most approximate orthogonal table and perform horizontal merging or pruning adjustments.

[0086] Each column of the orthogonal array is mapped to a parameter, and each level value in the column is mapped to an equivalence class representative value of the parameter. Each row of the orthogonal array is decoded into a complete test case, which contains the specific values ​​of all parameters.

[0087] After performing orthogonal defect combination in the orthogonal array, add benchmark test cases where all parameters take preset values ​​and extreme test cases where all parameters take extreme outliers simultaneously to supplement the peripheral scenarios not covered by orthogonal defect combination.

[0088] In some exemplary embodiments, the process of the automated execution and monitoring module performing kernel status monitoring and anomaly detection further includes:

[0089] Deadlock detection includes: starting a watchdog timer before each API call; if the API call does not return within the set maximum allowed time, it is judged as a suspected deadlock, and the status of all tasks, the resources they hold, and the resources they are waiting for are recorded.

[0090] Memory out-of-bounds detection includes: performing integrity checks on the preset memory sentinel region before and after API calls; if the characteristic value of the preset memory sentinel region changes, it is determined to be a memory out-of-bounds write, and the corrupted memory address range is recorded.

[0091] Kernel panic detection includes registering a kernel panic callback hook. When the RTOS kernel triggers the error handling process, the error context information is captured through the kernel panic callback hook. The error context information includes the exception type, error code, address of the faulty instruction, and call stack backtracking. A kernel panic can also be referred to as a kernel error.

[0092] Abnormal restart detection includes: maintaining test progress markers in a non-volatile storage area, and checking whether the test progress markers match the expected next test case number at each startup. If they do not match, it is determined that the previous API call caused an unexpected system restart.

[0093] In some exemplary embodiments, the process of the automated execution and monitoring module determining test results and generating reports further includes:

[0094] For test cases called with non-target parameters, if the API returns a predefined error code and the kernel status is normal, the test is considered to have passed, indicating that the kernel correctly rejected the illegal input; the non-target parameter can also be called an illegal parameter.

[0095] For test cases that are called with non-target parameters, if the API does not return an error code but executes normal logic, or causes at least one of the following abnormal events to be detected: deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart, the test is deemed to have failed, indicating that the kernel failed to safely intercept illegal input.

[0096] For benchmark test cases invoked with target parameters, if the API returns the expected result and the kernel status is normal, the test is considered to have passed; if the API returns an exception or the kernel status is abnormal, the test is considered to have failed. The target parameter can also be referred to as a valid parameter.

[0097] In some exemplary embodiments, the RTOS kernel API includes at least one of the following: management APIs (taskSpawn, taskDelete, taskSuspend, taskResume), semaphore APIs (semCreate, semTake, semGive, semDelete), message queue APIs (msgQCreate, msgQSend, msgQReceive, msgQDelete), timer APIs (wdCreate, wdStart, wdCancel), and memory management APIs (memPartAlloc, memPartFree). All of the aforementioned APIs are used to perform parametric equivalence class partitioning and orthogonal defect combination testing.

[0098] In some exemplary embodiments, the multidimensional equivalence class partitioning process of the parameter equivalence class partitioning module is as follows: Figure 2 As shown. See also Figure 2 This embodiment uses the RTOS semaphore acquisition function semTake(SEM_ID semId, int timeout) as an example to illustrate the process of multidimensional equivalence class partitioning.

[0099] The semTake() function takes two arguments:

[0100] - semId (of type SEM_ID, essentially a pointer): Semaphore identifier

[0101] - timeout (int type): The timeout period for waiting.

[0102] The equivalence class partitioning for each parameter is shown in Table 1 below:

[0103]

[0104] The parameter equivalence class partitioning module extracts the parameter list and data type of each function by parsing the header file and interface documentation of the target API. For pointer type parameters, it partitions into equivalence classes such as valid pointer, null pointer, dangling pointer, and unaligned pointer; for integer type length or timeout parameters, it partitions into equivalence classes such as normal value, zero value, negative value, upper bound value, and excessively large value; the same applies to flag and enumeration parameters. A representative value is selected from each equivalence class for subsequent combinations. The key here is that the partitioning rules are determined according to two dimensions: the data type of the parameter and its semantic role (buffer pointer, timeout parameter, permission flag, etc.), rather than a simple numerical partitioning.

[0105] In some exemplary embodiments, the orthogonal table mapping process of the orthogonal defect combination module is as follows: Figure 3 As shown. See also Figure 3This embodiment continues to use semTake() as an example to illustrate the process of orthogonal array mapping and test case generation.

[0106] The `semTake()` function takes two arguments, each with five equivalence classes, thus requiring a two-factor, five-level orthogonal array. We choose an L_25 (5^2) orthogonal array (or a more concise L_10 mixed orthogonal array), mapping the arguments to columns and the equivalence classes to levels.

[0107] Table 2 below shows some examples of test cases generated by orthogonal combinations:

[0108]

[0109] The orthogonal defect combination module selects the orthogonal table with the fewest rows from a pre-built library of mixed-level orthogonal tables based on the number of parameters k and the number of equivalence classes for each parameter. Each column of the orthogonal table maps to a parameter, and each level value maps to a representative value of an equivalence class for that parameter. Each row represents a complete test case. Taking a scenario with 5 parameters as an example: 2 parameters each have 5 equivalence classes, resulting in 25 Cartesian combinations. Using an orthogonal array, the number of test cases can be controlled to around 10, while ensuring that the equivalence class combination between any two parameters occurs at least once. In addition to orthogonal combinations, the module also adds all valid baseline test cases and all extreme outlier test cases to supplement extreme scenarios that orthogonal combinations may not include.

[0110] In some exemplary embodiments, the four-layer anomaly detection mechanism of the automated execution and monitoring module is as follows: Figure 4 As shown. See also Figure 4 This embodiment illustrates the four-layer anomaly detection mechanism of the automated test execution framework.

[0111] The test execution framework is deployed on the target RTOS platform and runs as a high-priority test task. For each test case, the execution framework proceeds in the following sequence:

[0112] 1. Before the call: Start the watchdog timer, collect kernel state snapshots (task list, semaphore status, memory pool status), and set up the memory sentinel region.

[0113] 2. During the call: The target API call is executed, and the watchdog timer continues counting.

[0114] 3. After the call: cancel the watchdog timer, verify the integrity of the memory sentinel region, compare the kernel state snapshot, and check the API return value.

[0115] The automated execution and monitoring module is deployed on the target RTOS platform, calling APIs one by one as high-priority test tasks. Kernel state snapshots are collected before and after each call. The monitoring process includes four checkpoints: a watchdog timer checks if the API returns within a specified time (deadlock detection); a memory sentinel verifies if the API has written to unauthorized memory; a kernel panic callback hook captures panic information; and progress markers in non-volatile memory determine if an unexpected restart has occurred. For test cases that trigger exceptions, the module finds the minimum parameter combination causing the exception by progressively reducing the parameter combinations, helping developers pinpoint the root cause of the problem.

[0116] In some exemplary embodiments, a complete testing process using the semTake() API as an example is provided. See [link / reference] Figure 5 Taking the semaphore acquisition function semTake(SEM_ID semId, int timeout) as an example, the complete test process from parameter partitioning to defect discovery is demonstrated.

[0117] Assuming 10 test cases are generated through orthogonal combination, the execution result is as follows:

[0118] - TC-01 (Valid ID, 1000): API returns OK → √ Passed.

[0119] - TC-02 (Valid ID, -100): API returns ERROR → √ Pass (Correct rejection of negative timeout).

[0120] - TC-03 (NULL, 0): API returns ERROR → √ Pass (correctly rejects null pointers).

[0121] - TC-04 (dangling pointer, WAIT_FOREVER): Triggered kernel panic → × failed.

[0122] - TC-05 (0xDEADBEEF, 0x7FFFFFFF): Watchdog timeout (deadlock) → × Failure.

[0123] - ...

[0124] Perform anomaly attribution analysis on failed TC-04 and TC-05:

[0125] - TC-04 Attribution: Dangling pointer is the key parameter triggering kernel panic (changing timeout to other values ​​still results in the error).

[0126] - TC-05 cause: Deadlock caused by combination of illegal address + extreme timeout (illegal address alone returns normal error code, but when combined with extreme timeout, it triggers kernel scheduling exception).

[0127] In some exemplary embodiments, anomaly attribution analysis includes associating each failed test case with its parameter equivalence class combination to identify the minimum parameter combination that caused the anomaly, thereby assisting developers in locating the cause of the defect. Specifically, anomaly attribution analysis can employ a binary search-based minimum anomaly parameter combination identification strategy, with the following specific steps:

[0128] Step A1: For a test case that is determined to fail, record its complete parameter combination as P = {p1,p2, ..., pk}, where k is the total number of parameters of the API, and pi is the current value of the i-th parameter (which may be a valid value or an invalid equivalence class value).

[0129] Step A2: Identify the parameters in the parameter set P that take illegal equivalence class values, and denote them as the abnormal parameter set S = {ps1, ps2, ..., psm}, where m is the number of parameters that take illegal equivalence class values, and m ≤ k; if m = 1, then the single abnormal parameter is the smallest parameter combination that causes the abnormality, and the attribution analysis ends.

[0130] Step A3: If m ≥ 2, divide the abnormal parameter set S into two equal parts. and the second half ;in, Indicates rounding down;

[0131] Step A4: Construct a verification test case V_front, where the parameters in S_front retain their original illegal values, while all parameters in S_back are replaced with the representative values ​​of the legal equivalence classes of the corresponding parameters, and the remaining parameters that originally have legal values ​​remain unchanged; execute the verification test case V_front and observe whether the same type of abnormal event is still triggered;

[0132] Step A5: Construct a verification test case V_back, where the parameters in S_back retain their original illegal values, while all parameters in S_front are replaced with the representative values ​​of the legal equivalence classes of the corresponding parameters, and the remaining parameters that originally have legal values ​​remain unchanged; execute the verification test case V_back and observe whether the same type of abnormal event is still triggered;

[0133] Step A6: Based on the execution results of steps A4 and A5, recursively reduce according to the following rules:

[0134] If V_front triggers an exception but V_back does not, then the exception parameter set is reduced to S_front, and steps A3 to A6 are repeated for S_front to perform recursive binary search.

[0135] If V_back triggers an exception but V_front does not, then the set of exception parameters is reduced to S_back, and steps A3 to A6 are repeated for S_back to perform recursive binary search.

[0136] If both V_front and V_back trigger exceptions, it means that S_front and S_back each independently contain parameters that can trigger exceptions. Recursively perform binary search on S_front and S_back independently, and finally take the set with the fewest elements among all independent recursive results as the minimum exception parameter combination.

[0137] If neither V_front nor V_back triggers an exception, it means the exception is triggered by the interaction of parameters in S_front and S_back, and a combination of both is required to reproduce the exception. In this case, S_front and S_back are further reduced by binary search, and unnecessary parameters are gradually removed using cross-validation: each exception parameter in S is replaced with a valid value and the test is re-executed. If the exception disappears after replacing a parameter, then that parameter is a necessary parameter for triggering the exception and is retained; if the exception still exists after replacement, then that parameter is unnecessary and is excluded. After traversing all exception parameters, the remaining set of necessary parameters is the minimum combination of parameters that causes the exception.

[0138] Step A7: When the set of abnormal parameters is reduced to the point where it cannot be reduced further (i.e., the abnormality disappears after each parameter in the set is replaced with a valid value), output the current set as the final minimum combination of abnormal parameters, and record this combination along with the abnormality type and fault scene information in the defect location information of the test report;

[0139] Using the above binary search strategy, the number of times the verification test cases for anomaly attribution analysis are executed is on the order of O(m × log m), where m is the initial number of anomaly parameters. Compared with the O(2^m) order of verifying all parameter subsets one by one, the binary search significantly reduces the execution overhead of attribution analysis and can complete the defect localization of multi-parameter API failure test cases within a reasonable time.

[0140] In summary, this invention provides a robustness testing system for the kernel interface of an airborne real-time operating system. This system employs a multi-dimensional equivalence class partitioning model for parameter classification, systematically enumerating six categories—legal values, boundary values, illegal values, null pointers, extreme values, and permission exceptions—based on two dimensions: data type and semantic role. This avoids the problem of overlooking exception categories that is common in manual, experience-based design. Furthermore, this system utilizes orthogonal lists for parameter combination, mathematically ensuring that the equivalence class combination between any two parameters is tested at least once. Moreover, at the test execution level, this system simultaneously deploys four mechanisms: watchdog deadlock detection, memory sentinel out-of-bounds detection, kernel panic callback capture, and non-volatile progress marker restart detection, making the result determination independent of the API return value itself. Additionally, the system's exception attribution analysis can identify the smallest parameter combination that triggers the exception, narrowing the defect investigation scope from the entire input set to specific values ​​of one or two parameters, thus shortening the developer's troubleshooting time. Furthermore, the structured test reports generated by this system include coverage statistics, details of abnormal test cases, and defect location information, which can serve as objective evidence of DO-178C robustness testing activities, meeting the requirements of airworthiness certification for test sufficiency and traceability. Therefore, the entire process from parameter classification, test case generation, test execution to result determination and report output requires no manual intervention. The same API set can be repeatedly regressed, ensuring consistency of test results across different batches. Thus, this system establishes multi-dimensional equivalence classes for each parameter of the RTOS kernel API according to data type and semantic role, and uses orthogonal lists to combine the representative values ​​of each parameter's equivalence classes, ensuring that every pairwise abnormal combination of parameters is covered with a smaller number of test cases. During the test execution phase, four mechanisms—watchdog timer, memory sentinel verification, kernel panic callback hook, and non-volatile progress markers—monitor the kernel status, detecting deadlock, memory out-of-bounds errors, kernel panic, and unexpected restarts, automatically determining test results and generating test reports. Therefore, this system addresses the problems of low efficiency in manual code writing and lack of systematic random testing by proposing a robust testing scheme based on an orthogonal defect model. First, each API parameter is partitioned into equivalence classes according to parameter type and semantic role (covering legal values, boundary values, illegal values, null pointers, extreme timeouts, and permission flag anomalies). Then, an orthogonal array is used to combine the equivalence classes of each parameter. Mathematically, the orthogonal array guarantees that each pair of equivalence classes for each pair of parameters appears at least once, thus achieving pairwise interactive coverage with far fewer test cases than a complete Cartesian combination. Simultaneously, during the test execution phase, the API is automatically invoked on the target RTOS platform, the kernel status is monitored, and a test report is automatically generated after determining the results.

[0141] This embodiment provides a method for testing the robustness of an airborne real-time operating system kernel interface, including the following steps:

[0142] S110, parameter equivalence class partitioning, includes: parsing the interface definition of the RTOS kernel API, extracting all parameters and their data types for each API function, and establishing an equivalence class partitioning model for each parameter according to its data type and semantic role.

[0143] S120, Orthogonal defect combination, includes: selecting an orthogonal table based on the number of parameters of each API function and the number of equivalence classes of each parameter, and combining the representative values ​​of the equivalence classes of each parameter according to the column mapping relationship of the orthogonal table to generate a robust test case set for each API; wherein, each test case contains at least one set of specific values ​​for all parameters of the corresponding API function.

[0144] S130, Automated test execution, includes: sequentially calling the corresponding RTOS kernel API for each test case and collecting kernel state snapshots through a pre-deployed or real-time automated test execution framework.

[0145] S140, Kernel Status Monitoring and Anomaly Detection, includes: real-time monitoring of the RTOS kernel's running status, detection of abnormal events, and recording, when an abnormal event is detected, the test case parameter combination that triggered the corresponding abnormal event, the abnormal type, and a kernel status snapshot; wherein, abnormal events include at least one of deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart.

[0146] S150, Test Result Determination and Report Generation, includes: automatically determining the status of each test case based on API return values ​​and kernel status monitoring results, and generating a structured robustness test report based on the status of all test cases. The robustness test report includes test coverage statistics, summary of abnormal test cases, anomaly attribution analysis, and defect location information; wherein, the status of the test case is either pass or fail.

[0147] In some exemplary embodiments, the process of establishing the equivalence class partitioning model in step S110 may include:

[0148] For pointer type parameters, the established equivalence classes include: valid pointer pointing to the target memory region, null pointer (NULL), dangling pointer pointing to a freed memory region, unaligned pointer, and pointer to a read-only memory region.

[0149] For integer type length or size parameters, the established equivalence classes include: normal values ​​within the target range, zero values, negative values ​​(for signed types), upper bound values ​​of the data type, and outliers outside the target range. The target range can also be referred to as the legal range.

[0150] For the timeout parameter, the established equivalence classes include: normal timeout value, zero timeout (immediate return), infinite wait flag value, negative timeout value (for signed types), and extreme timeout value close to the upper bound of the data type.

[0151] For permission flags or option parameters, the established equivalence classes include: target flag value or flag combination, all-zero flag, all-one flag (all bits set), non-target flag value (value not belonging to the predefined flag set), and mutually exclusive flag. Among them, the target flag value can also be called the valid flag value, and the non-target flag value can also be called the invalid flag value.

[0152] For enumeration type parameters, the established equivalence classes include: all target enumeration items, integer values ​​outside the enumeration range, and values ​​one above the upper bound of the enumeration type. The target enumeration items can also be referred to as valid enumeration items.

[0153] In some exemplary embodiments, the orthogonal defect combination process in step S120 may include: counting the number of parameters k of the target API function and the number of equivalence classes for each parameter, determining the number of factors and the number of levels for each factor in the orthogonal table; selecting the orthogonal table from a pre-built mixed-level orthogonal table library that satisfies k factors, the number of levels for each factor is not less than the number of equivalence classes for the corresponding parameter, and the smallest number of rows; when no exactly matching orthogonal table exists, selecting the closest orthogonal table and performing level merging or trimming adjustments; mapping each column of the orthogonal table to a parameter, mapping each level value in the column to a representative value of an equivalence class of that parameter, and decoding each row of the orthogonal table into a complete test case, the complete test case containing the specific values ​​of all parameters; after orthogonal defect combination of the orthogonal table, adding benchmark test cases where all parameters take preset values ​​and extreme test cases where all parameters simultaneously take extreme outliers to supplement the marginal scenarios not covered by orthogonal defect combination.

[0154] In some exemplary embodiments, the kernel status monitoring and anomaly detection process in step S140 may include:

[0155] Deadlock detection includes: starting a watchdog timer before each API call; if the API call does not return within the set maximum allowed time, it is judged as a suspected deadlock, and the status of all tasks, the resources they hold, and the resources they are waiting for are recorded.

[0156] Memory out-of-bounds detection includes: performing integrity checks on the preset memory sentinel region before and after API calls; if the characteristic value of the preset memory sentinel region changes, it is determined to be a memory out-of-bounds write, and the corrupted memory address range is recorded.

[0157] Kernel panic detection includes: registering kernel panic callback hooks; when the RTOS kernel triggers the error handling process, the kernel panic callback hooks capture error context information, including exception type, error code, fault instruction address, and call stack backtracking.

[0158] Abnormal restart detection includes: maintaining test progress markers in a non-volatile storage area, and checking whether the test progress markers match the expected next test case number at each startup. If they do not match, it is determined that the previous API call caused an unexpected system restart.

[0159] In some exemplary embodiments, the process of determining test results and generating reports may include:

[0160] For test cases called with non-target parameters, if the API returns a predefined error code and the kernel status is normal, the test is considered to have passed, indicating that the kernel correctly rejected the illegal input; the non-target parameter can also be called an illegal parameter.

[0161] For test cases that are called with non-target parameters, if the API does not return an error code but executes normal logic, or causes at least one of the following abnormal events to be detected: deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart, the test is deemed to have failed, indicating that the kernel failed to safely intercept illegal input.

[0162] For benchmark test cases invoked with target parameters, if the API returns the expected result and the kernel status is normal, the test is considered to have passed; if the API returns an exception or the kernel status is abnormal, the test is considered to have failed. The target parameter can also be referred to as a valid parameter.

[0163] In some exemplary embodiments, anomaly attribution analysis includes associating each failed test case with its parameter equivalence class combination to identify the minimum parameter combination that caused the anomaly, thereby assisting developers in locating the cause of the defect. Specifically, anomaly attribution analysis can employ a binary search-based minimum anomaly parameter combination identification strategy, with the following specific steps:

[0164] Step A1: For a test case that is determined to fail, record its complete parameter combination as P = {p1,p2, ..., pk}, where k is the total number of parameters of the API, and pi is the current value of the i-th parameter (which may be a valid value or an invalid equivalence class value).

[0165] Step A2: Identify the parameters in the parameter set P that take illegal equivalence class values, and denote them as the abnormal parameter set S = {ps1, ps2, ..., psm}, where m is the number of parameters that take illegal equivalence class values, and m ≤ k; if m = 1, then the single abnormal parameter is the smallest parameter combination that causes the abnormality, and the attribution analysis ends.

[0166] Step A3: If m ≥ 2, divide the abnormal parameter set S into two equal parts. and the second half ;in, Indicates rounding down;

[0167] Step A4: Construct a verification test case V_front, where the parameters in S_front retain their original illegal values, while all parameters in S_back are replaced with the representative values ​​of the legal equivalence classes of the corresponding parameters, and the remaining parameters that originally have legal values ​​remain unchanged; execute the verification test case V_front and observe whether the same type of abnormal event is still triggered;

[0168] Step A5: Construct a verification test case V_back, where the parameters in S_back retain their original illegal values, while all parameters in S_front are replaced with the representative values ​​of the legal equivalence classes of the corresponding parameters, and the remaining parameters that originally have legal values ​​remain unchanged; execute the verification test case V_back and observe whether the same type of abnormal event is still triggered;

[0169] Step A6: Based on the execution results of steps A4 and A5, recursively reduce according to the following rules:

[0170] If V_front triggers an exception but V_back does not, then the exception parameter set is reduced to S_front, and steps A3 to A6 are repeated for S_front to perform recursive binary search.

[0171] If V_back triggers an exception but V_front does not, then the set of exception parameters is reduced to S_back, and steps A3 to A6 are repeated for S_back to perform recursive binary search.

[0172] If both V_front and V_back trigger exceptions, it means that S_front and S_back each independently contain parameters that can trigger exceptions. Recursively perform binary search on S_front and S_back independently, and finally take the set with the fewest elements among all independent recursive results as the minimum exception parameter combination.

[0173] If neither V_front nor V_back triggers an exception, it means the exception is triggered by the interaction of parameters in S_front and S_back, and a combination of both is required to reproduce the exception. In this case, S_front and S_back are further reduced by binary search, and unnecessary parameters are gradually removed using cross-validation: each exception parameter in S is replaced with a valid value and the test is re-executed. If the exception disappears after replacing a parameter, then that parameter is a necessary parameter for triggering the exception and is retained; if the exception still exists after replacement, then that parameter is unnecessary and is excluded. After traversing all exception parameters, the remaining set of necessary parameters is the minimum combination of parameters that causes the exception.

[0174] Step A7: When the set of abnormal parameters is reduced to the point where it cannot be reduced further (i.e., the abnormality disappears after each parameter in the set is replaced with a valid value), output the current set as the final minimum combination of abnormal parameters, and record this combination along with the abnormality type and fault scene information in the defect location information of the test report;

[0175] Using the above binary search strategy, the number of times the verification test cases for anomaly attribution analysis are executed is on the order of O(m × log m), where m is the initial number of anomaly parameters. Compared with the O(2^m) order of verifying all parameter subsets one by one, the binary search significantly reduces the execution overhead of attribution analysis and can complete the defect localization of multi-parameter API failure test cases within a reasonable time.

[0176] In some exemplary embodiments, the RTOS kernel API includes at least one of the following: management APIs (taskSpawn, taskDelete, taskSuspend, taskResume), semaphore APIs (semCreate, semTake, semGive, semDelete), message queue APIs (msgQCreate, msgQSend, msgQReceive, msgQDelete), timer APIs (wdCreate, wdStart, wdCancel), and memory management APIs (memPartAlloc, memPartFree). All of the aforementioned APIs are used to perform parametric equivalence class partitioning and orthogonal defect combination testing.

[0177] In summary, this invention provides a robustness testing method for the kernel interface of an airborne real-time operating system. This method employs a multi-dimensional equivalence class partitioning model for parameter classification, systematically enumerating six categories—legal values, boundary values, illegal values, null pointers, extreme values, and permission exceptions—based on two dimensions: data type and semantic role. This avoids the problem of overlooking exception categories that is easily encountered in manual, experience-based design. Furthermore, this method utilizes orthogonal arrays for parameter combination, mathematically ensuring that the equivalence class combination between any two parameters is tested at least once. Moreover, at the test execution level, this method simultaneously deploys four mechanisms: watchdog deadlock detection, memory sentinel out-of-bounds detection, kernel panic callback capture, and non-volatile progress marker restart detection, making the result determination independent of the API return value itself. Additionally, the anomaly attribution analysis of this method can identify the smallest parameter combination that triggers the anomaly, narrowing the scope of defect investigation from the entire set of inputs to specific values ​​of one or two parameters, thus shortening the developer's troubleshooting time. Furthermore, the structured test report generated by this method includes coverage statistics, details of abnormal test cases, and defect location information, which can serve as objective evidence of DO-178C robustness testing activities, meeting the requirements of airworthiness certification for test sufficiency and traceability. Therefore, this method requires no manual intervention throughout the entire process from parameter classification, test case generation, test execution to result determination and report output. The same API set can be repeatedly regressed for testing, ensuring consistency of test results across different batches. Thus, this method establishes multi-dimensional equivalence classes for each parameter of the RTOS kernel API according to data type and semantic role, and uses orthogonal arrays to combine the representative values ​​of each parameter's equivalence classes, ensuring that every pairwise abnormal combination of parameters is covered with a smaller number of test cases. During the test execution phase, four mechanisms—watchdog timer, memory sentinel verification, kernel panic callback hook, and non-volatile progress markers—monitor the kernel status, detecting deadlock, memory out-of-bounds errors, kernel panic, and unexpected restarts, automatically determining test results and generating test reports. Therefore, this method addresses the problems of low efficiency in manual test writing and lack of systematic random testing by proposing a robust testing scheme based on an orthogonal defect model. It first divides each API parameter into equivalence classes according to parameter type and semantic role (covering legal values, boundary values, illegal values, null pointers, extreme timeouts, and permission flag anomalies). Then, it uses an orthogonal array to combine the equivalence classes of each parameter. Mathematically, the orthogonal array guarantees that each pair of equivalence classes for each pair of parameters appears at least once, thus achieving pairwise interactive coverage with far fewer test cases than a Cartesian combination. Simultaneously, during the test execution phase, the API is automatically invoked on the target RTOS platform, the kernel status is monitored, and a test report is automatically generated after determining the results.

[0178] It should be noted that the airborne real-time operating system kernel interface robustness testing system provided in the above embodiments and the airborne real-time operating system kernel interface robustness testing method provided in the above embodiments belong to the same concept. In practical applications, the airborne real-time operating system kernel interface robustness testing system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the steps described in the above method, and no limitation is imposed here. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0179] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for testing the robustness of an airborne real-time operating system kernel interface, characterized in that, The method includes the following steps: The parameter equivalence class partitioning includes: parsing the interface definition of the RTOS kernel API, extracting all parameters and their data types for each API function, and establishing an equivalence class partitioning model for each parameter according to its data type and semantic role; Orthogonal defect combination includes: selecting an orthogonal table based on the number of parameters of each API function and the number of equivalence classes of each parameter, and combining the representative values ​​of the equivalence classes of each parameter according to the column mapping relationship of the orthogonal table to generate a robust test case set for each API; wherein each test case contains at least one specific set of values ​​for all parameters of the corresponding API function; Automated test execution includes: sequentially calling the corresponding RTOS kernel API for each test case and collecting kernel state snapshots through a pre-deployed or real-time automated test execution framework; Kernel status monitoring and anomaly detection include: real-time monitoring of the RTOS kernel's running status, detection of abnormal events, and recording, when an abnormal event is detected, the test case parameter combination that triggered the corresponding abnormal event, the anomaly type, and a kernel status snapshot; wherein, the abnormal event includes at least one of deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart; Test result determination and report generation include: automatically determining the status of each test case based on API return values ​​and kernel status monitoring results, and generating a structured robustness test report based on the status of all test cases. The robustness test report includes test coverage statistics, summary of abnormal test cases, anomaly attribution analysis, and defect location information; wherein, the status of the test case is either pass or fail.

2. The robustness testing method for the airborne real-time operating system kernel interface according to claim 1, characterized in that, The process of establishing the equivalence class partitioning model includes: For pointer type parameters, the established equivalence classes include: valid pointer pointing to the target memory region, null pointer, dangling pointer pointing to a freed memory region, unaligned pointer, and pointer to a read-only memory region; For integer type length or size parameters, the established equivalence classes include: normal values ​​within the target range, zero values, negative values, upper bound values ​​of the data type, and outlier values ​​outside the target range; For the timeout parameter, the established equivalence classes include: normal timeout value, zero timeout, infinite wait flag value, negative timeout value, and extreme timeout value close to the upper bound of the data type. For permission flags or option parameters, the established equivalence classes include: target flag value or flag combination, all-zero flag, all-one flag, non-target flag value, and mutually exclusive flag; For enumeration type parameters, the established equivalence classes include: all target enumeration items, integer values ​​outside the enumeration range, and values ​​with the upper bound of the enumeration type plus one.

3. The robustness testing method for the airborne real-time operating system kernel interface according to claim 1, characterized in that, The process of combining orthogonal defects also includes: The number of parameters k and the number of equivalence classes for each parameter in the target API function are statistically analyzed to determine the number of factors and the number of levels for each factor in the orthogonal array. Select the orthogonal table that satisfies the following conditions from the pre-built mixed-level orthogonal table library: k factors, the number of levels of each factor is not less than the number of equivalence classes of the corresponding parameters, and the smallest number of rows. When there is no exactly matching orthogonal table, select the most approximate orthogonal table and perform horizontal merging or pruning adjustments. Each column of the orthogonal array is mapped to a parameter, and each level value in the column is mapped to an equivalence class representative value of the parameter. Each row of the orthogonal array is decoded into a complete test case, which contains the specific values ​​of all parameters. After performing orthogonal defect combination in the orthogonal array, add benchmark test cases where all parameters take preset values ​​and extreme test cases where all parameters take extreme outliers simultaneously to supplement the peripheral scenarios not covered by orthogonal defect combination.

4. The robustness testing method for the airborne real-time operating system kernel interface according to claim 1, characterized in that, The kernel status monitoring and anomaly detection process also includes: Deadlock detection includes: starting a watchdog timer before each API call; if the API call does not return within the set maximum allowed time, it is judged as a suspected deadlock, and the status of all tasks, the resources they hold, and the resources they are waiting for are recorded. Memory out-of-bounds detection includes: performing integrity checks on the preset memory sentinel region before and after API calls; if the characteristic value of the preset memory sentinel region changes, it is determined to be a memory out-of-bounds write, and the corrupted memory address range is recorded. Kernel panic detection includes: registering a kernel panic callback hook; when the RTOS kernel triggers the error handling process, the kernel panic callback hook captures error context information, including exception type, error code, fault instruction address, and call stack backtracking. Abnormal restart detection includes: maintaining test progress markers in a non-volatile storage area, and checking whether the test progress markers match the expected next test case number at each startup. If they do not match, it is determined that the previous API call caused an unexpected system restart.

5. The robustness testing method for the airborne real-time operating system kernel interface according to claim 1, characterized in that, The process of determining test results and generating reports also includes: For test cases called with non-target parameters, if the API returns a predefined error code and the kernel status is normal, the test is considered to have passed. For test cases called with non-target parameters, if the API does not return an error code but executes normal logic, or causes at least one of the following abnormal events to be detected: deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart, the test is considered to have failed. For benchmark test cases called with target parameters, if the API returns the expected result and the kernel status is normal, the test is considered to have passed; if the API returns an exception or the kernel status is abnormal, the test is considered to have failed.

6. The robustness testing method for the airborne real-time operating system kernel interface according to claim 1 or 5, characterized in that, The anomaly attribution analysis includes a minimum anomaly parameter combination identification strategy based on binary search. This strategy associates each failed test case with its parameter equivalence class combination to identify the minimum parameter combination causing the anomaly, thus pinpointing the cause of the defect. The specific steps are as follows: Step A1: For a test case that is determined to fail, record its complete parameter combination as P = {p1, p2,..., pk}, where k is the total number of parameters of the API and pi is the current value of the i-th parameter; Step A2: Identify the parameters that take illegal equivalence class values ​​in the parameter set P, and denote them as the abnormal parameter set S = {ps1, ps2, ..., psm}, where m is the number of parameters that take illegal equivalence class values, m ≤ k; if m = 1, then the single abnormal parameter is the smallest parameter combination that causes the abnormality, and the attribution analysis ends. Step A3: If m ≥ 2, divide the abnormal parameter set S into two equal parts. and the second half ;in, Indicates rounding down; Step A4: Construct a verification test case V_front, where the parameters in S_front retain their original illegal values, while all parameters in S_back are replaced with the representative values ​​of the legal equivalence classes of the corresponding parameters, and the remaining parameters that originally have legal values ​​remain unchanged; execute the verification test case V_front and observe whether the same type of abnormal event is still triggered; Step A5: Construct a verification test case V_back, where the parameters in S_back retain their original illegal values, while all parameters in S_front are replaced with the representative values ​​of the legal equivalence classes of the corresponding parameters, and the remaining parameters that originally have legal values ​​remain unchanged; execute the verification test case V_back and observe whether the same type of abnormal event is still triggered; Step A6: Based on the execution results of steps A4 and A5, recursively reduce according to the following rules: If V_front triggers an exception but V_back does not, then the exception parameter set is reduced to S_front, and steps A3 to A6 are repeated for S_front to perform recursive binary search. If V_back triggers an exception but V_front does not, then the set of exception parameters is reduced to S_back, and steps A3 to A6 are repeated for S_back to perform recursive binary search. If both V_front and V_back trigger exceptions, it means that S_front and S_back each independently contain parameters that can trigger exceptions. Recursively perform binary search on S_front and S_back independently, and finally take the set with the fewest elements among all independent recursive results as the minimum exception parameter combination. If neither V_front nor V_back triggers an exception, it means the exception is triggered by the interaction of parameters in S_front and S_back, and a combination of both is required to reproduce the exception. In this case, S_front and S_back are further reduced by binary search, and unnecessary parameters are gradually removed using cross-validation: each exception parameter in S is replaced with a valid value and the test is re-executed. If the exception disappears after replacing a parameter, then that parameter is a necessary parameter for triggering the exception and is retained; if the exception still exists after replacement, then that parameter is unnecessary and is excluded. After traversing all exception parameters, the remaining set of necessary parameters is the minimum combination of parameters that causes the exception. Step A7: When the set of abnormal parameters is reduced to the point where it cannot be reduced further (i.e., the abnormality disappears after each parameter in the set is replaced with a valid value), output the current set as the final minimum combination of abnormal parameters, and record this combination, along with the abnormality type and fault scene information, in the defect location information of the test report.

7. The robustness testing method for the airborne real-time operating system kernel interface according to claim 1, characterized in that, The RTOS kernel API includes at least one of the following: management API, semaphore API, message queue API, timer API, and memory management API.

8. A robustness testing system for the kernel interface of an airborne real-time operating system, characterized in that, The system includes: The parameter equivalence class partitioning module is used to parse the interface definition of the RTOS kernel API, extract all parameters and their data types for each API function, and establish an equivalence class partitioning model for each parameter according to its data type and semantic role. The orthogonal defect combination module is used to select an orthogonal table based on the number of parameters of each API function and the number of equivalence classes of each parameter, and combine the representative values ​​of the equivalence classes of each parameter according to the column mapping relationship of the orthogonal table to generate a robust test case set for each API; wherein, each test case contains at least one specific set of values ​​for all parameters of the corresponding API function; The automated execution and monitoring module is used to sequentially call the corresponding RTOS kernel API for each test case through a pre-deployed or real-time automated test execution framework, and to collect kernel state snapshots; and to monitor the running status of the RTOS kernel in real time, detect whether abnormal events occur, and when abnormal events are detected, record the test case parameter combination, exception type, and kernel state snapshot that triggered the corresponding abnormal event; wherein the abnormal events include at least one of deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart; and to automatically determine the status of each test case based on the API return value and kernel state monitoring results, and generate a structured robustness test report based on the status of all test cases, wherein the robustness test report includes test coverage statistics, abnormal test case summary, abnormal attribution analysis, and defect location information; wherein the status of the test cases is either pass or fail.

9. The airborne real-time operating system kernel interface robustness testing system according to claim 8, characterized in that, The orthogonal defect combination module further includes the following process for performing orthogonal defect combination: The number of parameters k and the number of equivalence classes for each parameter in the target API function are statistically analyzed to determine the number of factors and the number of levels for each factor in the orthogonal array. Select the orthogonal table that satisfies the following conditions from the pre-built mixed-level orthogonal table library: k factors, the number of levels of each factor is not less than the number of equivalence classes of the corresponding parameters, and the smallest number of rows. When there is no exactly matching orthogonal table, select the most approximate orthogonal table and perform horizontal merging or pruning adjustments. Each column of the orthogonal array is mapped to a parameter, and each level value in the column is mapped to an equivalence class representative value of the parameter. Each row of the orthogonal array is decoded into a complete test case, which contains the specific values ​​of all parameters. After performing orthogonal defect combination in the orthogonal array, add benchmark test cases where all parameters take preset values ​​and extreme test cases where all parameters take extreme outliers simultaneously to supplement the peripheral scenarios not covered by orthogonal defect combination.

10. The airborne real-time operating system kernel interface robustness testing system according to claim 8, characterized in that, The process of kernel status monitoring and anomaly detection performed by the automated execution and monitoring module also includes: Deadlock detection includes: starting a watchdog timer before each API call; if the API call does not return within the set maximum allowed time, it is judged as a suspected deadlock, and the status of all tasks, the resources they hold, and the resources they are waiting for are recorded. Memory out-of-bounds detection includes: performing integrity checks on the preset memory sentinel region before and after API calls; if the characteristic value of the preset memory sentinel region changes, it is determined to be a memory out-of-bounds write, and the corrupted memory address range is recorded. Kernel panic detection includes: registering a kernel panic callback hook; when the RTOS kernel triggers the error handling process, the kernel panic callback hook captures error context information, including exception type, error code, fault instruction address, and call stack backtracking. Abnormal restart detection includes: maintaining test progress markers in non-volatile storage areas and checking whether the test progress markers are consistent with the expected next test case number at each startup. If they are inconsistent, it is determined that the previous API call caused an unexpected system restart. The process of determining test results and generating reports by the automated execution and monitoring module also includes: For test cases called with non-target parameters, if the API returns a predefined error code and the kernel status is normal, the test is considered to have passed. For test cases called with non-target parameters, if the API does not return an error code but executes normal logic, or causes at least one of the following abnormal events to be detected: deadlock, memory out-of-bounds access, kernel panic, and unexpected system restart, the test is considered to have failed. For benchmark test cases called with target parameters, if the API returns the expected result and the kernel status is normal, the test is considered to have passed; if the API returns an exception or the kernel status is abnormal, the test is considered to have failed.

Images