Aviation airborne software test quality evaluation method, equipment and medium
By constructing a coverage-defect density correlation model and a defect propagation path diagram, the problems of multi-source data fusion and defect propagation in airborne software testing were solved, achieving efficient quality evaluation and resource scheduling, and improving the scientific nature and credibility of the test.
Patent Information
- Application Number
- CN202511272270.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-08
- Publication Date
- 2026-01-23
AI Technical Summary
Existing airborne software testing methods cannot effectively integrate multi-source data, lack defect propagation mechanisms, produce unintuitive quality evaluation results, and lack the ability to predict future risks, resulting in unreasonable allocation of testing resources and difficulty in translating evaluation results into optimization actions.
A coverage-defect density correlation model is constructed to generate a defect propagation path map. A three-layer quality evaluation index system is established, and combined with dynamic weight adjustment and defect prediction mechanism, multi-source data fusion, defect propagation path display and dynamic quality status evaluation are realized.
It achieves deep fusion of multi-source data and dynamic display of defect propagation paths, improving the scientificity and credibility of quality evaluation, enhancing defect localization efficiency and the targeting of testing resources, and supporting a comprehensive understanding of the overall testing situation and optimized resource allocation.
Smart Images
Figure CN121387709A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of aviation software testing, in particular to an aviation airborne software testing quality evaluation method, device and medium. BACKGROUND
[0002] With the rapid development of avionics systems towards high integration, high complexity and high safety, aviation airborne software plays an irreplaceable role in key fields such as flight control, avionics integration, task management, etc. The quality of software is directly related to the operation safety and task reliability of the aircraft, therefore, a comprehensive, scientific and systematic testing quality evaluation of airborne software is the core link to ensure the safe operation of aviation equipment.
[0003] Currently, the testing of aviation airborne software mainly relies on traditional means such as coverage rate statistics, test case verification and defect statistical analysis. Although these methods support the evaluation of testing sufficiency and defect situation to some extent, they still have many technical limitations, especially in data integration, dynamic tracking and visual evaluation, which are difficult to meet the high timeliness, high precision and high credibility requirements of quality evaluation under complex systems.
[0004] Existing testing quality evaluation methods are mostly based on single-dimensional static data, making it difficult to realize the fusion and collaborative analysis of multi-source data. For example, coverage rate data, test case data and defect data are usually stored in independent testing tools or management platforms, lacking a unified data structure and semantic interface, making it difficult to effectively correlate modeling. This "information island" problem directly leads to one-sided quality evaluation conclusions, which are difficult to reflect the actual testing risks.
[0005] Current most testing evaluation systems do not consider the propagation and system dependency of defects in the testing process. Airborne software is usually composed of multiple highly correlated functional modules, and any defect in a module may affect multiple downstream modules due to call relationship, data dependency or interface interaction. Traditional methods lack the ability to model and display the propagation path of defects, and cannot reveal the transmission path and potential diffusion range of defects from the source point to the impact point, which often leads to inaccurate problem positioning and unclear regression test targets in actual projects.
[0006] In terms of display and explanation of testing results, traditional methods mostly output in the form of static reports, logs or tables, lacking interactivity, intuitiveness and timeliness. Testers need to switch between multiple tools, manually organize information, greatly increasing cognitive burden and communication cost, and it is difficult to quickly grasp the overall quality and evolution trend of the current software.
[0007] More importantly, existing methods generally lack the ability to predict future quality risks. Test work often faces limited resources and time constraints, and test managers need to focus on high-risk areas first. Without a prediction mechanism based on historical data, test resources cannot be effectively concentrated on modules that are most likely to expose defects, resulting in wasted test resources or blind spots in testing.
[0008] In this context, some industry practices attempt to introduce data analysis methods to assist quality evaluation, such as using coverage-driven regression testing, using defect distribution to predict potential risk areas, etc. However, these methods are mostly at the local optimization level, lack a unified architecture design and comprehensive modeling ability across data domains, and have not yet formed a complete quality evaluation system.
[0009] Aircraft software testing also faces the problem of "separation of evaluation and control". The quality evaluation results are not fed back in time to the test strategy and resource scheduling, making it difficult to translate evaluation conclusions into test optimization actions. For example, in some projects, although high-defect-density modules are identified, test case distribution is not adjusted in time, resulting in ineffective coverage of problems.
[0010] In addition, although DO-178C, ARINC 653, and other aviation software standards have strict requirements for testing activities, they do not provide a unified and systematic quality evaluation method, and project teams often rely on experience to develop test strategies, lacking scientific basis and quantitative support. SUMMARY
[0011] The present application provides an aircraft onboard software testing quality evaluation method, device and medium, aiming to solve the technical problems of existing test evaluation methods, such as the inability to fuse multi-source data, the lack of defect propagation mechanism, and the non-intuitive presentation of quality status.
[0012] The technical solution adopted by the present application to solve its technical problems is to provide an aircraft onboard software testing quality evaluation method, comprising the following steps:
[0013] Constructing a coverage-defect density correlation model based on multi-source test data fusion;
[0014] Constructing a module dependency graph and generating a defect propagation path graph in combination with defect source information;
[0015] Based on the coverage-defect density correlation model and the defect propagation path graph, dynamically evaluating the quality of aircraft onboard software testing.
[0016] The coverage-defect density correlation model based on multi-source test data fusion is constructed, specifically including:
[0017] Extracting code coverage data, test case data, and defect data from the software test tool chain;
[0018] The code coverage data, the test case data and the defect data are fused by a data fusion algorithm, and the fused data is analyzed to establish a coverage-defect density correlation model to determine the quantitative correlation between the coverage and the defect density.
[0019] The code coverage data includes statement coverage, branch coverage and path coverage; the test case data includes test case pass rate, failure rate and execution time; and the defect data includes defect quantity, severity level and distribution position.
[0020] The data fusion algorithm is a weighted fusion method, and when the code coverage data, the test case data and the defect data are fused, dynamic weights are assigned to the code coverage data, the test case execution data and the defect data according to different software development stages; wherein, the test case data is given the highest weight in the requirement analysis stage; the code coverage data is given the highest weight in the coding implementation stage; and the defect data is given the highest weight in the test verification stage.
[0021] The construction module generates a defect propagation path graph in combination with defect source information, and specifically includes:
[0022] The calling relationship between software modules is parsed, and a module dependency graph is constructed according to the parsing result;
[0023] Based on historical defect records and current defect trigger chains, the complete path of defects from the generation module to the propagation module is tracked to obtain a defect propagation path;
[0024] The software modules on the defect propagation path are represented by color coding to represent the severity level of the defect, and a defect propagation path graph is obtained.
[0025] The defect propagation path graph supports interactive operations, and the interactive operations include at least one of the following operations:
[0026] Clicking a software module displays detailed information about defects related to the software module;
[0027] Dragging the timeline to view the defect propagation state changes in different software development stages;
[0028] Filtering and displaying defect propagation paths of a certain severity level or type.
[0029] The three-layer quality evaluation index system includes a basic index layer quality evaluation index system, a derived index layer quality evaluation index system and a comprehensive index layer quality evaluation index system, evaluation indexes of the basic index layer quality evaluation index system include code coverage, defect density and test case pass rate, evaluation indexes of the derived index layer quality evaluation index system include defect convergence rate, defect repair efficiency and high-risk module concentration, and evaluation indexes of the comprehensive index layer quality evaluation index system include software quality index, module risk level and system reliability prediction value; the three-layer quality evaluation index system adopts a dynamic weight adjustment mechanism, and automatically adjusts the weight of each index according to different development stages of software.
[0030] The aviation airborne software test quality evaluation method further includes reading current test data, extracting feature variables of the current test data, inputting the feature variables into a trained defect prediction model, and obtaining a software module that may have defects, a risk level of an uncovered area, and a number and level of defects that may be found in the next test cycle.
[0031] The technical scheme adopted by the present application to solve its technical problems is to provide an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize the steps of the aviation airborne software test quality evaluation method.
[0032] The technical scheme adopted by the present application to solve its technical problems is to provide a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to realize the steps of the aviation airborne software test quality evaluation method.
[0033] Advantages
[0034] Compared with the prior art, the present application has the following advantages and positive effects:
[0035] The present application breaks the information barrier between traditional test tools by constructing a coverage-defect density correlation model and deeply integrating code coverage, test case execution data and defect information, realizes quantitative modeling of the logical relationship between test sufficiency and defect distribution, and improves the scientificity and reliability of the evaluation results.
[0036] The present application introduces a defect propagation path diagram to dynamically present the propagation process of defects from the source point to the affected modules, clearly reveals the high-risk path and the position of the key modules in the system, and significantly improves the defect positioning efficiency and the scheduling pertinence of test resources.
[0037] The application supports automatic adjustment of weights of various evaluation indexes based on project stages, and can update the evaluation model and visual view in real time with test data, ensuring that the test quality evaluation result has timeliness and evolution. The application displays the quality state through a multi-dimensional graphical interface, so that the tester can fully understand the overall situation of the test without relying on complex analysis, effectively supporting test process management and decision-making. BRIEF DESCRIPTION OF DRAWINGS
[0038] Figure 1 is a flowchart of the aviation airborne software test quality evaluation method of the first embodiment of the application;
[0039] Figure 2 is a schematic diagram of constructing a coverage-defect density correlation model in the first embodiment of the application;
[0040] Figure 3 is a defect propagation path diagram in the first embodiment of the application. DETAILED DESCRIPTION
[0041] The application will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the application and not to limit the scope of the application. In addition, it should be understood that those skilled in the art can make various modifications or changes to the application after reading the content taught by the application, and these equivalent forms also fall within the scope defined by the appended claims of the application.
[0042] The first embodiment of the application relates to an aviation airborne software test quality evaluation method, as shown in Figure 1 , comprising the following steps:
[0043] Step 1, constructing a coverage-defect density correlation model based on multi-source test data fusion. As shown in Figure 2 , this step is specifically as follows:
[0044] First, code coverage data, test case data and defect data are extracted from the software test tool chain, wherein the code coverage data includes statement coverage, branch coverage and path coverage, which reflects the inspection degree of the test on the code logic; the test case data includes test case pass rate, failure rate and execution time, which reflects the effectiveness and execution efficiency of the test case; the defect data includes defect quantity, severity level and distribution position, which is used to identify problem density and distribution. The extraction process can be integrated into the existing CI / CD process to realize automatic acquisition and update.
[0045] Then, the data fusion engine fuses the code coverage data, the test case data and the defect data by a data fusion algorithm. In the embodiment, the data fusion algorithm can adopt a weighted fusion algorithm. When fusing, different weights are assigned to the code coverage data, the test case data and the defect data according to the current software development stage. If the current software development stage is the requirement analysis stage, the highest weight is assigned to the test case data. If the current software development stage is the coding implementation stage, the highest weight is assigned to the code coverage data. If the current software development stage is the test verification stage, the highest weight is assigned to the defect data. The standardized data set is formed by the weighted fusion.
[0046] Finally, the fused data is analyzed to establish a coverage-defect density correlation model to determine the quantitative correlation between the coverage and the defect density. In the embodiment, statistical analysis techniques such as linear regression or principal component analysis can be adopted to construct a multivariate statistical model, thereby establishing the coverage-defect density correlation model. The coverage-defect density correlation model established by this method has self-learning ability and can automatically adjust parameters according to data updates, thereby realizing dynamic adaptation and continuous optimization.
[0047] The coverage-defect density correlation model established by the embodiment supports dynamic adjustment of the weights of the data in different software development stages, can be fitted to the actual evolution process of the test, and thereby improves the accuracy and reliability of the software test quality evaluation.
[0048] Step 2, construct a module dependency graph, and generate a defect propagation path graph in combination with the defect source information.
[0049] To visually show the propagation path and the influence range of the defect in the system, a defect propagation path graph is constructed in this step. The defect propagation path graph is generated based on the calling relationship between software modules and the defect trigger chain, and specifically as follows:
[0050] First, the calling relationship between software modules is analyzed, and a module dependency graph is constructed according to the analysis result. In this way, the calling paths between the software modules are determined;
[0051] Then, based on the historical defect records and the current defect trigger chain, the complete path of the defect from the generation module to the propagation module is tracked to obtain the defect propagation path. In this way, the defect source node, the propagation link and the finally affected software module are labeled.
[0052] Finally, the software modules on the defect propagation path are represented by color coding to represent the severity level of the defect, and the defect propagation path graph is obtained (see Figure 3 ). The color identification of the defect severity level can dynamically update the change of the defect over time, which is helpful for analyzing the evolution process of the defect.
[0053] The defect propagation path graph constructed in this step supports interactive operation. The tester can click a specific software module to view detailed information about defects related to the software module, or drag the time axis to view the change in the defect propagation state at different software development stages, or filter and display defects of a certain severity level or type.
[0054] It can be found that, by establishing a dependency graph between software modules, the embodiment traces the process of defect propagation from the source to each software module, records the propagation path and triggering conditions, and presents the defect impact path in a graphical manner, uses color to identify the defect level, supports interactive operation and dynamic state refresh, and enhances the defect tracing capability of software testing.
[0055] Step 3: Based on the coverage-defect density correlation model and the defect propagation path graph, the quality of the aviation onboard software testing is dynamically evaluated.
[0056] This step can integrate the constructed coverage-defect density correlation model and defect propagation path graph to develop an interactive dynamic visualization platform, realize intuitive software quality evaluation, for example: provide a quality score dashboard for rapid overall quality evaluation; display a risk module heat map and a defect propagation dynamic graph to highlight the problem area; support users to perform deep problem tracing and analysis through filtering conditions and time axis control.
[0057] This step establishes a three-layer quality evaluation index system to comprehensively reflect the effectiveness of the testing work and the system quality when dynamically evaluating the quality of the aviation onboard software testing. The three-layer quality evaluation index system includes a basic index layer quality evaluation index system, a derived index layer quality evaluation index system, and a comprehensive index layer quality evaluation index system.
[0058] The evaluation indexes of the basic index layer quality evaluation index system include code coverage, defect density, and test case pass rate; the evaluation indexes of the derived index layer quality evaluation index system include defect convergence rate, defect repair efficiency, and high-risk module concentration; and the evaluation indexes of the comprehensive index layer quality evaluation index system include software quality index, module risk level, and system reliability prediction value.
[0059] The three-layer quality evaluation index system can use a dynamic weight adjustment mechanism to automatically adjust the weights of each index according to different stages of the software development life cycle, for example: in the early development stage, the code coverage index in the basic index layer quality evaluation index system is emphasized; in the middle testing stage, defect-related indexes in the derived index layer quality evaluation index system are emphasized; and in the pre-release stage, the software quality index in the comprehensive index layer quality evaluation index system is emphasized.
[0060] Therefore, the embodiment constructs a hierarchical dynamic software quality evaluation index system, all evaluation indexes are uniformly displayed through a visual interface, and the index system supports a weight dynamic adjustment mechanism based on a life cycle stage, so that the index weights can be automatically adjusted according to the stage attention focus, the software quality quantitative evaluation from the local to the global is realized, the user is helped to comprehensively master the software test quality status, and the quality bottleneck and the high-risk module are quickly identified.
[0061] The aviation airborne software test quality evaluation method of the embodiment further includes a defect prediction mechanism. The defect prediction mechanism is specifically as follows.
[0062] The historical test data and defect data are acquired, the characteristic variables of the historical test data are extracted, the defect data is taken as a label, a data set is established, the data set is divided into a training set and a test set, the machine learning model is trained by using the training set, the machine learning model can be a decision tree model, a neural network model or a Bayesian network model, after the training is completed, the trained model is tested by using the test set, if the test is passed, a defect prediction model is obtained. When the defect prediction is performed, the current test data is acquired, the characteristic variables of the current test data are extracted, and the characteristic variables are input into the trained defect prediction model, so that the software module that may have defects, the risk level of the uncovered area, and the number and level of defects that may be found in the next test cycle are obtained. The prediction result can assist a test team in optimizing resource allocation, adjusting test priorities and test case sets, thereby improving the pertinence and overall efficiency of the test, and realizing the quality guarantee mode change from "post-event analysis" to "pre-event prevention".
[0063] Therefore, the embodiment can predict the potential defect position, the uncovered risk area and the number and level of defects that may be exposed in the future test cycle, and improve the test resource allocation efficiency and the forward-looking of the test strategy.
[0064] The second embodiment of the application relates to an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the aviation airborne software test quality evaluation method of the first embodiment when the computer program is executed.
[0065] The third embodiment of the application relates to a computer readable storage medium, and a computer program is stored on the computer readable storage medium, and the steps of the aviation airborne software test quality evaluation method of the first embodiment are implemented when the computer program is executed by a processor.
[0066] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage etc.) embodying computer readable program code.
[0067] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the application. It should be understood that each flow and / or block in the flowchart and / or block diagrams, and a combination of flows and / or blocks in the flowchart and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowchart and / or block diagrams of the flowchart and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified in the flowchart and / or block diagram.
[0068] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction method, which implements the functions specified in the flowchart and / or block diagrams of the flowchart and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified in the flowchart and / or block diagram.
[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operational steps are performed on the computer or other programmable data processing apparatus to produce a computer implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a means for implementing the functions specified in the flowchart and / or block diagrams of the flowchart and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the function specified in the flowchart and / or block diagram.
[0070] The above description is merely one specific implementation of the application, but the protection scope of the application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which should be covered by the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.
Claims
1. An airborne software testing quality evaluation method, characterized by, The method comprises the following steps: constructing a coverage-defect density correlation model based on multi-source test data fusion; constructing a module dependency graph and generating a defect propagation path graph in combination with defect source information; performing dynamic evaluation on the test quality of the aviation airborne software based on the coverage-defect density correlation model and the defect propagation path graph.
2. The airborne software testing quality evaluation method according to claim 1, characterized in that, The construction of the coverage-defect density correlation model based on multi-source test data fusion specifically comprises: extracting code coverage data, test case data and defect data from a software test tool chain; performing fusion processing on the code coverage data, test case data and defect data through a data fusion algorithm, performing correlation analysis on the data after the fusion processing, and establishing a coverage-defect density correlation model to determine the quantitative correlation between coverage and defect density.
3. The airborne software testing quality evaluation method according to claim 2, characterized in that, The code coverage data comprises statement coverage, branch coverage and path coverage; the test case data comprises test case pass rate, failure rate and execution time; The defect data comprises defect quantity, severity level and distribution position.
4. The airborne software testing quality evaluation method according to claim 2, characterized in that, The data fusion algorithm is a weighted fusion method, and when performing fusion processing on the code coverage data, test case data and defect data, dynamic weights are assigned to the code coverage data, test case execution data and defect data according to different software development stages; wherein the test case data is given the highest weight in the requirement analysis stage; the code coverage data is given the highest weight in the coding implementation stage; and the defect data is given the highest weight in the test verification stage.
5. The airborne software testing quality evaluation method of claim 1, wherein, The construction of the module dependency graph and the generation of the defect propagation path graph in combination with the defect source information specifically comprise: analyzing the calling relationship between software modules and constructing a module dependency graph according to the analysis result; tracking the complete path of defects from the generation module to the propagation module based on historical defect records and current defect trigger chains, obtaining a defect propagation path; using color coding to represent the severity level of defects on the software modules in the defect propagation path, and obtaining a defect propagation path graph.
6. The airborne software testing quality evaluation method of claim 1, wherein, The defect propagation path graph supports interactive operations, and the interactive operations at least include one of the following operations: clicking a software module to display detailed information about defects related to the software module; dragging a time axis to view the defect propagation state changes in different software development stages; filtering and displaying defect propagation paths of a specific severity level or type.
7. The airborne software testing quality evaluation method of claim 1, wherein, The three-layer quality evaluation index system includes a basic index layer quality evaluation index system, a derived index layer quality evaluation index system and a comprehensive index layer quality evaluation index system. The evaluation indexes of the basic index layer quality evaluation index system include code coverage, defect density and test case pass rate. The evaluation indexes of the derived index layer quality evaluation index system include defect convergence rate, defect repair efficiency and high-risk module concentration. The evaluation indexes of the comprehensive index layer quality evaluation index system include software quality index, module risk level and system reliability prediction value. The three-layer quality evaluation index system adopts a dynamic weight adjustment mechanism to automatically adjust the weights of the indexes according to different development stages of the software.
8. The airborne software testing quality evaluation method of claim 1, wherein, Further comprising: reading current test data, and extracting feature variables of the current test data, inputting the feature variables into a trained defect prediction model to obtain a software module that may have defects, a risk level of an uncovered area, and a number and level of defects that may be found in a next round of test cycles.
9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the airborne software test quality evaluation method according to any one of claims 1-8.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the airborne software test quality evaluation method according to any one of claims 1-8.