Airborne software security modeling and automatic testing method

Abstract

The embodiment of the invention provides an airborne software security modeling and automatic testing method, which comprises the following steps of: extracting security property description required by security testing by utilizing a security property extraction module according to the field to which airborne software belongs, and meanwhile, designing a SysML (System Modeling Language) model, uploading an XML (Extensible Markup Language) file and security property description of the SysML model to an automatic model extension module; the model automatic expansion module expands an XML file of the SysML model according to the uploaded security property description, and supplements the node type and path information of the SysML model to generate an expanded model; and the test case automatic generation module generates a comprehensive test path and test data of the expanded model according to the expanded model output by the model automatic expansion module. According to the scheme, through the security property extraction module, the model automatic extension module and the test case automatic generation module, a security test is added, and the test case generation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software security, and in particular to a method for airborne software security modeling and automated testing. Background Art

[0002] With the deepening of informatization and the rapid iteration of software engineering technology, the application scope of software in the fields of aviation, aerospace, electronics, shipbuilding, etc. is becoming more and more extensive. Especially in the field of military aircraft, the role played by software is becoming more and more important. Taking the F-22 and F-35 aircraft as examples, the software source code exceeds 7 million lines, realizing more than 80% of the functions of the whole aircraft, responsible for completing core key tasks such as flight control, mission planning, navigation identification, etc., and has become the most core part of the aviation equipment system. Its security level is crucial.

[0003] Airborne software systems are typical complex embedded systems with operating characteristics such as "frequent interface interactions, close coupling between software and hardware, complex mission scenarios, and strict timing constraints". The failure mechanism is relatively complex, and safety verification work faces huge challenges. Standard software safety verification work basically follows the "system hazard analysis-software failure mode analysis-software safety design-software safety testing" process. By conducting system hazard analysis, the software hazard sources, causes and impacts can be clarified. On this basis, the software failure mode can be analyzed to obtain safety requirements. Developers design software safety elements based on these requirements and form analysis criteria to guide safety testing. The complex application scenarios and operating requirements of airborne software itself make the failure mechanism significantly different from general software. It is necessary to combine the safety characteristics of airborne software, carry out field safety verification, and fully understand the safety level of airborne software.

[0004] At present, the idea of ​​layer-by-layer functional decomposition development is mostly adopted in the process of model development. Model-driven development (MDD) is a high-level abstract development method with models as the main artifacts, and its application scope is gradually expanding. However, the current model-driven airborne software development work is basically carried out around "requirements analysis-design model-model verification-code generation-code testing-use and maintenance", in which the goal of model verification is mainly to ensure the integrity and functional correctness of the model, and the code testing stage mainly completes functional testing and safety testing. The airborne software safety analysis and verification at the model layer have not been fully carried out. The model layer focuses on functional safety verification, and safety verification is only carried out at the code layer, resulting in the test lagging behind the development process, and the cost of repairing safety defects is high. Therefore, it is necessary to study the safety verification of the SysML model layer of airborne software to ensure the safety and reliability of model design. In addition, the safety analysis process is too dependent on manual work, which also leads to a low degree of automation in test verification, and the test efficiency is difficult to meet the requirements of the development progress.

[0005] In summary, the following pain points still exist in the process of airborne software safety verification:

[0006] 1) Incomplete safety modeling: The application of model-driven development methods in the field of airborne software development mainly focuses on the rapid construction of demand models and automatic code generation. Insufficient consideration is given to airborne software safety issues during model design and conversion. Existing related methods mostly focus on typical safety properties, including resource occupation conflicts, functional logic safety, etc., and cannot cover the complex failure modes of airborne software. In addition, existing safety analysis mostly relies on the quality department to conduct code analysis, and comprehensive safety modeling analysis is not carried out in a timely manner at the model level, resulting in untimely safety demonstration and high subsequent maintenance costs.

[0007] 2) The safety testing process is not efficient: At present, when conducting safety testing, the test cases are designed by the testers based on the system hazards or software failure modes identified in the system hazard analysis and software failure mode analysis, and the content of the software safety test cases (including input, constraints, expected output, passing criteria, etc.) is determined. However, this process is mainly designed manually by testers, forming a set of safety test cases in the form of a table, which leads to omissions of test cases and incomplete coverage of safety analysis criteria, resulting in a lengthy testing process, low efficiency in exposing safety defects, and delays in the progress of airborne software development. Summary of the invention

[0008] In view of this, the present invention provides a method for airborne software security modeling and automated testing, which is based on a security property extraction module, a model automatic expansion module, and a test case automatic generation module to solve the technical problems of insufficient model layer security testing and low test case generation efficiency in the prior art. The method for airborne software security modeling and automated testing includes:

[0009] According to the field to which the airborne software belongs, the safety property extraction module is used to extract the safety property description required for safety testing. At the same time, the SysML model of the airborne software is designed, and the XML file of the SysML model of the airborne software and the safety property description are uploaded to the model automatic expansion module;

[0010] The model automatic extension module extends the XML file of the airborne software SysML model according to the uploaded safety property description, and supplements the node type and path information of the airborne software SysML model to generate the extended model;

[0011] The test case automatic generation module generates a comprehensive test path and test data for the extended model based on the extended model output by the model automatic extension module.

[0012] Furthermore, the safety property extraction module extracts safety properties applicable to the airborne software from four categories of safety properties, the four categories of safety properties including a value overlimit category, a data timing category, a data communication category, and a fault handling category.

[0013] Furthermore, the model automatic extension module extends the SysML model of the airborne software through the large language model.

[0014] Furthermore, the large language model is a Qwen2-72B-Instict model.

[0015] Furthermore, the learning rate of the Qwen2-72B-Instict model is set to 5e-05 and / or 1e-05, the number of training cycles of the Qwen2-72B-Instict model is set to 100 and / or 50, the optimizer of the Qwen2-72B-Instict model is AdamW, the learning rate scheduler type of the Qwen2-72B-Instict model is cosine annealing, and the drop rate of LoRA of the Qwen2-72B-Instict model is 0.1.

[0016] Furthermore, the test case automatic generation module generates hyperparameters for the genetic algorithm through a large language model and improves the individual mutation process.

[0017] Furthermore, the test case automatic generation module generates test paths through an edge-first depth traversal method and generates test data through a genetic algorithm improved by a large language model.

[0018] Furthermore, the extended model includes model elements required for testing the safety of the onboard software.

[0019] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above technical solutions adopted in the embodiments of this specification include at least:

[0020] The SysML model of airborne software is expanded to enhance the expressiveness of the model and cover safety testing requirements. At the same time, the automated expansion based on the large language model reduces user workload and realizes the automatic generation of test cases, further improving the efficiency of airborne software safety testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1A flow chart of a method for airborne software security modeling and automated testing provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of the classification of safety properties of the method for airborne software safety modeling and automated testing provided by an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of automatic model expansion of the method for airborne software security modeling and automated testing provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of edge-first traversal of a method for airborne software security modeling and automated testing provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of test data generation for the method for airborne software security modeling and automated testing provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0028] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0029] The present invention provides a method for airborne software security modeling and automated testing, which extracts the security properties of the airborne software, expands the airborne software SysML model using a large language model, generates test paths using an edge-first traversal method, and generates test data using a genetic algorithm improved by the large language model, thereby achieving the automation level of airborne software security analysis and further improving the efficiency of airborne software security testing.

[0030] like Figure 1 As shown, in an embodiment of the present invention, a method for airborne software security modeling and automated testing is implemented based on a security property extraction module, a model automatic expansion module, and a test case automatic generation module, and specifically includes:

[0031] Step S100: extract the security properties required for security testing using the security property extraction module according to the field to which the airborne software belongs, design the SysML model of the airborne software, and upload the SysML model XML file and the security property description to the model automatic expansion module;

[0032] Furthermore, in step S100, uploading the security properties and SysML model XML file to the model automatic expansion module includes:

[0033] Step S101 : The safety property extraction module extracts safety properties applicable to airborne software from four types of safety properties, including a value exceeding limit category, a data timing category, a data communication category, and a fault handling category.

[0034] Specifically, in this embodiment, safety-related specifications are extracted from safety standard documents such as DO-178C and APR 4754A, and the failure mode characteristics of the airborne software are systematically analyzed to form safety properties that support safety testing. The corresponding classification is as follows: Figure 2 shown.

[0035] Step S102: The model automatic expansion module reads the security property and SysML model XML file to complete the upload.

[0036] Step S200, the model automatic expansion module expands the SysML model XML file according to the uploaded security property description, supplements the model node type and path information, and the expanded model includes the model elements required for airborne software security testing;

[0037] Furthermore, in step S200, the large language model is a Qwen2-72B-Instict model. In order to make the large language model better adapt to this field, it is necessary to fine-tune the model to understand and generate complex models related to security. The LoRA (Low-Rank Adaptation) parameter efficient fine-tuning technology is adopted, and two different learning rates (5e-05 and 1e-05), number of training cycles (100 and 50), optimizer (AdamW), learning rate scheduler type (cosine annealing) and LoRA discard rate (0.1) are selected. A low rank structure is introduced in the weight matrix of the pre-trained model to better adapt to specific tasks or fields.

[0038] Furthermore, in step S200, Prompt can guide the model to understand the user's needs more accurately, significantly improve the relevance and quality of the generated content, use Prompt with context text and Prompt without context text as the input of the large model, compare the generation effect of the model respectively, and use the large language model to expand the model process. Figure 3 shown.

[0039] Step S300, the test case automatic generation module generates a comprehensive test path and test data for the extended SysML model generated in step S200 according to the extended model output by the model automatic extension module. The test case automatic generation module generates the test path by an edge-first depth traversal method and generates test data by using a genetic algorithm improved by a large language model. The genetic algorithm is an optimization algorithm based on natural selection and genetic mechanisms. The standard steps include initializing hyperparameters, generating an initial population, calculating individual fitness, individual mutation, individual crossover, selecting individuals, and determining whether the individuals meet the requirements.

[0040] Furthermore, the edge-first depth-first traversal method recursively traverses all nodes and edges in the graph, records all independent paths and edges traversed, starts from the starting node, and gradually goes deeper into each unvisited edge until all edges are visited. During the traversal process, the current path and visited edges are dynamically updated, and the backtracking mechanism is used to ensure the comprehensiveness and completeness of the traversal. The traversal algorithm is as follows: Figure 4 shown.

[0041] Furthermore, the large language model improved genetic algorithm method uses LLMs (large language models) to optimize the hyperparameter selection and individual mutation process of the genetic algorithm. By training LLMs to identify historical test data and corresponding algorithm performance data, the model can learn which hyperparameter settings are more effective in specific test scenarios and help automatically adjust the parameters of the genetic algorithm to meet different test requirements. At the same time, LLMs can propose more targeted mutation strategies based on a deep understanding of test data and system behavior, thereby improving the global search ability of the algorithm. The test data generation process is as follows: Figure 5 shown.

[0042] Furthermore, the method of improving the genetic algorithm using a large language model by introducing a large language model can significantly optimize the hyperparameter generation and individual mutation process of the genetic algorithm. The large language model can consider more comprehensive contextual information when generating hyperparameters through an in-depth understanding of the context, thereby improving the performance of the genetic algorithm. In addition, the large language model can retain and utilize the contextual relationship of the original data during the individual mutation process, ensuring that the mutated individual still meets the triggering conditions for the conversion.

[0043] Furthermore, a large language model is used to generate hyperparameters (POPULATIONIZE, CROSSOV-RATE, MUTATION-RATE, MAX GENERATES) for the genetic algorithm and to improve the individual mutation process, where POPULATIONIZE is the number of candidate solutions retained and processed by the algorithm in each generation, CROSSOV-RATE is the probability that two individuals (parents) in the genetic algorithm exchange gene fragments to generate new individuals (offspring), MUTATION-RATE is the probability of random mutation of individual genes in each generation, and MAX GENERATES is the maximum number of generations run by the genetic algorithm.

[0044] In summary, the embodiments of the present invention achieve the following technical effects:

[0045] The method for airborne software security modeling and automated testing of the embodiment of the present invention has higher development performance and efficiency, realizes the extraction of security properties, and expands the airborne software SysML model based on this, enhances the expressiveness of the model, covers security testing requirements, and reduces user workload based on the automated expansion of the large language model. At the same time, it realizes the automatic generation of test cases, further improving the efficiency of airborne software security testing.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for airborne software security modeling and automated testing, characterized in that: The method for airborne software security modeling and automated testing is implemented based on a security property extraction module, a model automatic expansion module, and a test case automatic generation module, and includes: According to the field to which the airborne software belongs, the safety property extraction module is used to extract the safety property description required for the safety test, and at the same time, the airborne software SysML model is designed, and the XML file of the airborne software SysML model and the safety property description are uploaded to the model automatic expansion module; The model automatic expansion module expands the XML file of the airborne software SysML model according to the uploaded safety property description, and supplements the node type and path information of the airborne software SysML model to generate an expanded model; The test case automatic generation module generates a comprehensive test path and test data of the extended model based on the extended model output by the model automatic extension module.

2. The method for airborne software security modeling and automated testing according to claim 1, characterized in that: The safety property extraction module extracts safety properties applicable to the onboard software from four categories of safety properties, wherein the four categories of safety properties include a value exceeding limit category, a data timing category, a data communication category, and a fault handling category.

3. The method for airborne software security modeling and automated testing according to claim 1, characterized in that: The model automatic expansion module expands the onboard software SysML model through a large language model.

4. The method for airborne software security modeling and automated testing according to claim 3, characterized in that: The large language model is the Qwen2-72B-Instict model.

5. The method for airborne software security modeling and automated testing according to claim 4, characterized in that: The learning rate of the Qwen2-72B-Instict model is set to 5e-05 and / or 1e-05, the number of training cycles of the Qwen2-72B-Instict model is set to 100 and / or 50, the optimizer of the Qwen2-72B-Instict model is AdamW, the learning rate scheduler type of the Qwen2-72B-Instict model is cosine annealing, and the drop rate of LoRA of the Qwen2-72B-Instict model is 0.

1.

6. The method for airborne software security modeling and automated testing according to claim 1, characterized in that: The test case automatic generation module generates hyperparameters for the genetic algorithm through a large language model and improves the individual mutation process.

7. The method for airborne software security modeling and automated testing according to claim 1, characterized in that: The test case automatic generation module generates a test path through an edge-first depth traversal method, and generates test data through a genetic algorithm improved by a large language model.

8. The method for airborne software security modeling and automated testing according to claim 1, characterized in that: The extended model includes model elements required for testing the safety of the onboard software.

Images