Embedded software defect detecting and filtering method, system and medium

Abstract

The invention relates to the field of embedded software research and development, and discloses an embedded software defect detecting and filtering method and system and a medium. According to the method, a software static analysis defect detection report is output by performing static analysis on an embedded software source code by adopting a method of combining software static defect detection and large-model intelligent defect alarm filtering; generating a cue word script according to the defect detection requirement of the embedded software; and filtering the software static analysis defect detection report by using a pre-trained large model according to the cue word script, and outputting a software model analysis defect detection report. By further analyzing and processing a static detection result, filtering and determining real and effective software defects, the effect of detecting and positioning real defect vulnerabilities of the embedded system software is improved, the quality and stability of the embedded system software are enhanced, and rapid iterative upgrading of the embedded system software is supported.

Description

Technical Field

[0001] This invention relates to the field of embedded software development, specifically to an embedded software defect detection and filtering method, system, and medium. Background Technology

[0002] To accurately describe the information in this article, the following conventions are hereby established:

[0003] Large models refer to models with billions to trillions of parameters, such as GPT-3 (175 billion parameters), GPT-4 (approximately 1.8 trillion parameters), and PaLM (540 billion parameters).

[0004] Existing embedded system software is characterized by complex scenarios, large management nodes, and massive amounts of interfaces and data. The software scale is increasing exponentially. In the process of software engineering applications, software defects are an unavoidable problem during development and maintenance. These defects affect the stability of embedded systems, potentially leading to security and economic losses for countries and enterprises. Currently, software defect detection methods mainly rely on manual code review or conventional static defect detection tools. This results in massive amounts of data, a high false positive rate, and often requires further manual verification. It is characterized by slow localization and low repair efficiency, becoming a burden for developers to maintain software quality and failing to meet the requirements of agile development in current embedded software. Summary of the Invention

[0005] To address the problems of low efficiency in manual code review and high false alarm rate and low location efficiency in existing software defect detection methods, and to achieve rapid location and efficient repair of software defects, this application proposes an embedded software defect detection and filtering method.

[0006] Firstly, an embedded software defect detection and filtering method is provided, adopting the following technical solution:

[0007] Perform static analysis on the embedded software source code and output a static analysis defect detection report.

[0008] Generate prompt scripts based on embedded software defect detection requirements;

[0009] Using a pre-trained large model, the software static analysis defect detection report is filtered according to the prompt word script, and a software model analysis defect detection report is output.

[0010] Furthermore, static analysis of embedded software source code includes syntax parsing, semantic parsing, abstract syntax tree generation, and value flow analysis.

[0011] Furthermore, the static analysis defect detection report includes defect category information, the file name of the defect, defect location information, and defect variable name.

[0012] Furthermore, when generating the prompt word script based on the embedded software defect detection requirements, the specific steps are as follows:

[0013] The prompt word structured framework template clearly defines the identity of the embedded software engineering analysis expert, identifies the target defects that need to be detected and filtered, adds example guidance, and provides test cases and descriptions of model input and output information.

[0014] Furthermore, the pre-trained large model used is Code Llama.

[0015] Furthermore, using a pre-trained large model, the software static analysis defect detection report is filtered according to the prompt word script, and the output software model analysis defect detection report is as follows:

[0016] A confidence filter value is set for the pre-trained large model. When the confidence probability of a defect is greater than or equal to the confidence filter value, the defect is retained; otherwise, it is filtered out.

[0017] Furthermore, the model analysis defect detection report removes false alarms and unimportant alarms from the static analysis defect detection report, and sorts the defects according to their severity and priority, including the confidence probability of each defect.

[0018] Secondly, an embedded software defect detection and filtering system is provided to implement the embedded software defect detection and filtering method described in the first aspect, comprising the following modules:

[0019] The static analysis engine module is used to perform static analysis on embedded software source code and output a static analysis defect detection report.

[0020] The large model defect filtering module relies on the underlying basic framework module and prompting engineering module to read the static analysis defect detection report, judge the defect information in the static analysis defect detection report, and output the defect confidence probability judgment result.

[0021] The underlying basic framework module of the model is the runtime dependency of the large model defect filtering function module, and is responsible for the training and inference process of the large model.

[0022] The prompting module is used to generate prompt scripts based on the needs of embedded software defect detection.

[0023] The confidence ranking generation module filters and organizes the defect confidence probabilities output by the large model defect filtering function module, and outputs a model analysis defect detection report.

[0024] An automated operation framework module is used to connect the various modules in the embedded software defect detection and filtering system and drive the execution of the defect detection and filtering process.

[0025] Thirdly, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, characterized in that, when the computer-executable instructions in the computer-readable storage medium are executed, the embedded software defect detection and filtering method as described in the first aspect is implemented.

[0026] The beneficial effects of this invention are as follows:

[0027] This application employs a method combining static software defect detection with intelligent defect alarm filtering based on a large-scale model. It supports the detection and localization of typical defects such as memory leaks and array out-of-bounds errors. It also supports further analysis, processing, filtering, and identification of real and valid software defects based on prompt scripts and a large-scale model. The method features scalable defect feature information input and an intelligent large-scale model that can be optimized and upgraded, enabling rapid defect localization and efficient repair, thus improving the agility of embedded software development. This approach can enhance the detection and localization of real defects and vulnerabilities in embedded system software, strengthen the quality and stability of embedded system software, and support rapid iterative upgrades of embedded system software. Attached Figure Description

[0028] Figure 1 This is a flowchart of the embedded software defect detection and filtering method according to an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of the working principle of the embedded software defect detection and filtering system according to an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram illustrating the working principle of the large model defect filtering function module in an embodiment of the present invention. Detailed Implementation

[0031] The present invention will now be described in further detail.

[0032] This application provides a method for embedded software defect detection and filtering, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0033] Step 1: Perform static analysis on the embedded software source code and output a static analysis defect detection report.

[0034] The method invokes a defect detection engine to perform static analysis on the input embedded software source code file. This typically includes common static defect analysis techniques such as syntax parsing, semantic parsing, AST (Abstract Syntax Tree) generation, and value flow analysis. Based on certain logical rules, it determines whether software defects exist in the scanned code and outputs relevant information such as defect category, defect filename, defect location (row and column numbers), and defect variable names. It supports the detection and localization of typical defects such as memory leaks and array out-of-bounds errors, generating an embedded software defect detection report. In addition to using self-developed syntax and semantic parsing tools, this method also employs open-source static analysis tools such as Infer and Saber, as well as the commercial tool Cobot, as implementations for certain engine types in this module. Furthermore, this method designs a standard defect detection report format and corresponding conversion tools, demonstrating its openness.

[0035] Step 2: Generate prompt scripts based on embedded software defect detection requirements.

[0036] Based on the software defect categories output by the defect detection engine, and combined with a pre-defined structured framework template for prompts, a prompt script is generated that can be executed by a large model. First, the model role is defined using the structured framework template, clearly identifying the user as an embedded software engineering analysis expert and further explaining the capabilities required for this role. Then, a scenario description is provided, where the structured framework template and software defect categories are combined to clarify the target defects to be detected and filtered in this large model operation, supplemented by certain detection constraints. Next, example guidance is added, providing test cases and descriptions of model input and output information, constraining the output format of the large model, and ultimately forming a prompt script for a specific embedded software defect.

[0037] The prompting script used in this embodiment is as follows:

[0038]

[0039] Step 3: Using a pre-trained large model, filter the static analysis defect report of the embedded software based on the prompt script, and output the model analysis defect detection report.

[0040] Use pre-trained large models, such as Code Llama and Qwen, to filter embedded software defect reports based on prompt scripts before outputting them.

[0041] This embodiment uses a pre-trained large model of Code Llama, implemented using the PyTorch framework. The deployment platform for the Llama large model is a Phytium server supplemented by two A100 GPUs. First, the Python and PyTorch software packages are installed using the apt tool to prepare the environment for the large model. Then, Code Llama version 70B is downloaded from GitHub, installed, and the Phytium platform is optimized and configured. Furthermore, the platform centrally manages the defect confidence output of the Code Llama pre-trained large model, setting confidence filter values ​​to filter embedded software static analysis defect reports. Defects are retained when the model's confidence probability for a defect is greater than or equal to the set confidence filter value; otherwise, they are filtered out. This identifies defect alerts or alerts that developers are more likely to be concerned about, filtering out false positives and unimportant alerts, thereby reducing the number of alerts from the static analysis tool and outputting the confidence probability of each defect. These defects can also be directly observed through a human-machine interface, and further adjustments can be made based on user judgment. Based on the severity and priority of the defects, the identified defects are sorted, and the embedded software model analysis defect report is output in a predetermined format so that developers can prioritize fixing serious and urgent defects.

[0042] This invention also provides an embedded software defect detection and filtering system, including a static detection engine module, a large model defect filtering function module, a model underlying basic framework module, a prompting engineering module, a confidence ranking generation module, and an automated operation framework module. The working principle of the embedded software defect detection and filtering system is as follows: Figure 2 As shown, the description is as follows:

[0043] After receiving the source code file of the embedded software under test, the static detection engine module calls the engine to perform static analysis on the input embedded software source code, outputting a software static analysis defect detection report. This report is then sent to the large model defect filtering module by the automated execution framework module. The large model defect filtering module, relying on the underlying basic framework module and the prompting engineering module, reads the software static analysis defect detection report, judges the defect-related information in the report, and outputs the confidence probability judgment result for each software defect to the confidence ranking generation module. The confidence ranking generation module filters and organizes the defect confidence probabilities output by the large model defect filtering module, outputting a software model analysis defect detection report. This report removes false positives and unimportant alarms from the static analysis defect detection report and sorts the defects according to their severity and priority, including the confidence probability of each defect. The defect detection and filtering ranking of embedded software is a series of automated software defect detection operations; the platform can automatically drive the operation after the user selects the input. This method combines a pre-trained large model to further confirm and filter static defects in embedded software using machine learning. The standard-defined interface has good openness, and static detection engines, pre-trained large models, etc. can be upgraded and replaced.

[0044] Each module in the embedded software defect detection and filtering system will be described in detail below:

[0045] The static analysis engine module, upon receiving the source code file of the embedded software under test, invokes an engine to perform static analysis on the input embedded software source code. This typically includes common static defect analysis techniques such as syntax parsing, semantic parsing, AST tree generation, and value flow analysis. It uses certain logical rules to determine whether software defects exist in the scanned code and outputs information related to the software defects, including the defect's filename, location information (row and column numbers), and variable names. It supports the detection and localization of typical defects such as memory leaks and array out-of-bounds errors, generating a static analysis defect detection report. In addition to using self-developed syntax and semantic parsing tools, this module also employs open-source static analysis tools such as Infer and Saber, as well as the commercial tool Cobot, as implementations for certain engine types. Furthermore, this method designs a standard defect detection report format and corresponding conversion tools, demonstrating openness. The embedded software defect detection report output by the static analysis engine module is driven by the automated execution framework module and sent to the large model defect filtering function module.

[0046] The large-scale model defect filtering module relies on the underlying framework module and the prompting engineering module to read the software static analysis defect detection report, and judges the defect-related information in the report, outputting the confidence probability judgment result for each software defect. The large-scale model defect filtering module uses pre-trained large models, such as Code Llama and Qwen, combined with the prompting scripts output by the prompting engineering module to analyze the embedded software defect report output by the static detection engine module. Based on the alarm characteristics reported by the static analysis tool, this module automatically identifies real defect alarms or alarms that developers are more likely to pay attention to, filtering out false alarms and unimportant alarms, thereby reducing the number of alarms from the static analysis tool, and outputting the confidence probability of each defect. The working principle of the large-scale model defect filtering module is as follows: Figure 3 As shown, the system includes input for a software static analysis defect detection report, a prompting script, a pre-trained large model, core samples, defect dataset construction, training dataset, model fine-tuning based on human feedback samples, model optimization, defect feature information, and standardized output of defect confidence scores. After the software static analysis defect detection report is input, the prompting script (specially designed) invokes the Code Llama pre-trained large model to read each defect feature from the report and determine its confidence score. The processing result is then sent to the confidence score ranking generation module to output standardized discrimination results. The pre-trained large model constructs the defect dataset using core samples, which are high-quality datasets that have undergone rigorous screening. By constructing the defect dataset, the basic capability boundaries of the model are established, ensuring the large model's ability to perform basic tasks such as software defect identification and confidence score determination. This dataset is formed by combining international standard software defect sets with existing data sets accumulated in the embedded systems field. The output judgment results are further confirmed manually. Samples that have passed manual review are added to the training sample set. The training dataset consists of core samples and a dataset that has been manually confirmed after automated defect detection by a large model. The pre-trained large model is retrained using the training set to optimize the model and adjust its parameters to better suit software defect detection in embedded environments. Defect detection and filtering ranking of embedded software is a series of automated software defect detection operations. The platform can automatically drive the operation after the user selects the input.

[0047] The underlying framework module of the model is the basic operating framework of the large model defect filtering function module, responsible for the training and inference process of the large model. Since this example uses Code Llama to pre-train the large model, it is built using the PyTorch framework to support the retraining and optimization of the large model in the large model defect filtering function module.

[0048] The prompting module is responsible for generating prompt scripts that conform to certain formatting requirements based on the embedded software defect detection needs. In this example, for instance... Figure 3As shown, users can further adjust and optimize the interface. The automated execution framework module drives the script to call the large model defect filtering function module, which can be defined and modified by the user.

[0049] The confidence ranking generation module filters and organizes the defect confidence probabilities output by the large model defect filtering module, outputting a defect detection report in a specific format. This module also ranks identified defects based on their severity and priority, allowing developers to prioritize the repair of critical and urgent defects, and then organizes the report output according to a defined format.

[0050] The automated operation framework module is used to automatically drive the entire defect detection and filtering process. In this embodiment, it is built using the Spring Boot + React technology stack. The specific operations such as pipeline communication between modules, model optimization, user input, and report output are implemented through coding, connecting the work of each module in this method.

[0051] Users obtain defect detection and filtering ranking reports for the embedded software under test through an automated running framework. If problems are found, the software is adjusted and the automated defect detection and filtering ranking process is repeated. If no problems are found, the testing process is completed.

[0052] This application also provides a computer-readable storage medium storing computer-executable instructions, wherein the embedded software defect detection and filtering method described in this application is implemented when the computer-executable instructions in the computer-readable storage medium are executed.

[0053] While the present invention has been disclosed above with reference to preferred embodiments, these embodiments are not intended to limit the invention. Any equivalent changes or modifications made without departing from the spirit and scope of the invention are also within the scope of protection of the invention. Therefore, the scope of protection of the present invention should be determined by the claims of this application.

Claims

1. A method for detecting and filtering defects in embedded software, characterized in that, Includes the following steps: Perform static analysis on the embedded software source code and output a static analysis defect detection report. Generate prompt scripts based on embedded software defect detection requirements; Using a pre-trained large model, the software static analysis defect detection report is filtered according to the prompt word script, and a software model analysis defect detection report is output.

2. The embedded software defect detection and filtering method according to claim 1, characterized in that, Static analysis of embedded software source code includes syntax parsing, semantic parsing, abstract syntax tree generation, and value flow analysis.

3. The embedded software defect detection and filtering method according to claim 1, characterized in that, The static analysis defect detection report includes defect category information, the file name of the defect, defect location information, and defect variable name.

4. The embedded software defect detection and filtering method according to claim 1, characterized in that, When generating a prompt word script based on the requirements for embedded software defect detection, the specific steps are as follows: The prompt word structured framework template clearly defines the identity of the embedded software engineering analysis expert, identifies the target defects that need to be detected and filtered, adds example guidance, and provides test cases and descriptions of model input and output information.

5. The embedded software defect detection and filtering method according to claim 1, characterized in that, The pre-trained large model used is Code Llama.

6. The embedded software defect detection and filtering method according to claim 1, characterized in that, When using a pre-trained large model to filter the software static analysis defect detection report based on the prompt word script, and outputting the software model analysis defect detection report, the specific steps are as follows: A confidence filter value is set for the pre-trained large model. When the confidence probability of a defect is greater than or equal to the confidence filter value, the defect is retained; otherwise, it is filtered out.

7. The embedded software defect detection and filtering method according to claim 1, characterized in that, The model analysis defect detection report removes false alarms and unimportant alerts from the static analysis defect detection report, and sorts the defects according to their severity and priority, including the confidence probability of each defect.

8. An embedded software defect detection and filtering system, used to implement the embedded software defect detection and filtering method as described in claim 1, characterized in that, Includes the following modules: The static analysis engine module is used to perform static analysis on embedded software source code and output a static analysis defect detection report. The large model defect filtering module relies on the underlying basic framework module and prompting engineering module to read the static analysis defect detection report, judge the defect information in the static analysis defect detection report, and output the defect confidence probability judgment result. The underlying basic framework module of the model is the runtime dependency of the large model defect filtering function module, and is responsible for the training and inference process of the large model. The prompting module is used to generate prompt scripts based on the needs of embedded software defect detection. The confidence ranking generation module filters and organizes the defect confidence probabilities output by the large model defect filtering function module, and outputs a model analysis defect detection report. An automated operation framework module is used to connect the various modules in the embedded software defect detection and filtering system and drive the execution of the defect detection and filtering process.

9. A computer-readable storage medium storing computer-executable instructions, characterized in that, The embedded software defect detection and filtering method as described in claims 1 to 5 is implemented when executing the computer-executable instructions in the computer-readable storage medium.

Images