Multi-agent collaborative HarmonyOS API fuzzy testing method, device and system and storage medium

Abstract

The invention discloses a multi-agent collaborative HarmonyOS API (Application Program Interface) fuzzy testing method, device and system and a storage medium. The method comprises the following steps: preprocessing a HarmonyOS development document; constructing a cue word input large model by utilizing the pre-processed document, optimizing a cue word template in combination with reinforcement learning, and integrating an Android example code to generate a corresponding driving code; performing grammar detection on the generated drive code to generate a high-quality seed bank; performing cyclic fuzzy testing by utilizing the seed bank, wherein the cyclic fuzzy testing comprises seed selection, variation, compiling operation, log collection and system screenshot; and performing vulnerability analysis based on logs and system screenshots generated in the loop fuzzy test process. According to the invention, the vulnerability discovery efficiency is improved, and the manual workload is reduced.

Description

Technical Field

[0001] This invention relates to the field of fuzz testing technology in cybersecurity, and in particular to a multi-agent collaborative HarmonyOS API fuzz testing method, apparatus, system, and storage medium. Background Technology

[0002] HarmonyOS is a mobile operating system with a microkernel architecture and a unified API framework, enabling seamless interaction between heterogeneous devices. As of 2025, it has been deployed on over one billion devices worldwide, including smartphones, tablets, IoT devices, smart TVs, and PCs, forming a massive user base and a rapidly expanding ecosystem. While this API layer provides cross-device collaboration and resource sharing capabilities, it also significantly expands the system's attack surface, introducing potential vulnerabilities that could threaten system security.

[0003] To ensure the security of the HarmonyOS API, one of the most effective methods is fuzzing. Fuzzing systematically generates and executes a large number of fuzzed drivers—containing unexpected, malformed inputs—to expose hidden implementation flaws, boundary violations, and memory corruption vulnerabilities. Over the past decade, fuzzing has proven extremely effective in practice, uncovering thousands of critical vulnerabilities in mainstream operating systems such as Linux, Windows, Android, and iOS.

[0004] Existing fuzzy driver generation methods can be broadly categorized into three types: graph-based generation, machine learning (ML)-based generation, and large language model (LLM)-based generation. In graph-based driver generation, static analysis, dynamic instrumentation, and symbolic execution are used to construct system execution graphs (including API call graphs, control flow graphs, and data flow graphs). These graphs are then traversed and analyzed to identify all reachable paths to the target API, extracting common call sequences and parameter dependencies to automatically generate the driver. In machine learning-based generation, example code and network protocol traffic are used as training datasets. Sequence-to-sequence (Seq2Seq) and generative adversarial networks (GANs) are employed to learn input sequence patterns, automatically generating fuzzy drivers. In LLM-based generation, background knowledge (including development documentation, test cases, or code knowledge graphs) and retrieval-enhanced hints are input into the LLM. The LLM then directly generates the fuzzy driver.

[0005] Although the above solutions are widely used in traditional operating systems, they cannot be directly applied to HarmonyOSAPI for the following reasons:

[0006] The effectiveness of fuzzy driving is low: the quality of the generated driving is limited by the fuzzer's ability to recover the target API's logical structure. For HarmonyOS, the current fuzzer has limited capabilities in this regard because (1) HarmonyOS is a relatively new operating system with few publicly available API call examples; (2) the documentation is not detailed enough, and key flowcharts and diagrams are embedded in the form of images, which hinders accurate automated semantic extraction; and (3) the text and diagrams in the documentation are intertwined, which complicates cross-modal semantic alignment.

[0007] Inefficiency in crash deduplication and classification: HarmonyOS's microkernel architecture and asynchronous APIs result in crash characteristics that often differ from those of traditional operating systems. Therefore, existing crash deduplication algorithms (mostly designed for traditional operating systems) may perform poorly on HarmonyOS—incorrectly merging crashes from different vulnerabilities or splitting multiple manifestations of the same vulnerability into different categories. Summary of the Invention

[0008] This invention proposes a multi-agent collaborative HarmonyOS API fuzzing method, apparatus, system, and storage medium, aiming to solve the problems of the lack of examples in the HarmonyOS API documentation and the lack of multimodal data processing in traditional methods.

[0009] On the one hand, to achieve the above objectives, this invention provides a multi-agent collaborative HarmonyOS API fuzzing method, comprising:

[0010] Preprocess the HarmonyOS development documentation;

[0011] We construct a large model for prompt word input using preprocessed documents, optimize prompt word templates using reinforcement learning, and integrate Android sample code to generate corresponding driver code.

[0012] The generated driver code undergoes syntax checking to produce a high-quality seed library;

[0013] Cyclic fuzz testing was performed using the seed library, including seed selection, mutation, compilation and execution, log collection, and system screenshots.

[0014] Vulnerability analysis was performed based on logs and system screenshots generated during the cyclic fuzzing process.

[0015] Preferably, the HarmonyOS development documentation is preprocessed, including: document filtering, multimodal data conversion, and permission data extraction;

[0016] The document filtering process is used to filter out irrelevant grammatical documents and retain interface documents related to HarmonyOS application development; the multimodal data conversion is used to convert multimodal visual information in the documents into text data; and the permission data extraction is used to generate project permission configuration files.

[0017] Preferably, the multimodal data conversion includes:

[0018] Extract each image from the document, input a multimodal large model to generate a text description, and insert the text description at the end of the original document.

[0019] Preferably, generating the corresponding driver code includes:

[0020] The optimal template is selected from a predefined prompt word template library based on reinforcement learning;

[0021] Recall Android sample code that matches the semantics of the current function and integrate it into the prompt word input model to generate the corresponding driver code.

[0022] Preferably, during the generation of the corresponding driver code, the agent selects the optimal template, the state data comes from the compilation feedback data, and the reward function is selected based on the compilation success rate, error repair difficulty, and execution efficiency optimization prompts.

[0023] Preferably, performing cyclic fuzz testing using the seed library includes:

[0024] Step 1: Initialize the energy value of the driver code in the seed library;

[0025] Step 2: Select seeds based on the energy values, and perform mutation operations on the selected seeds using a large model;

[0026] Step 3: Compile and run the mutated code, and collect logs and system screenshots simultaneously;

[0027] Step 4: Update the seed energy value based on the log analysis results, and iterate through steps 2-4.

[0028] Preferably, vulnerability analysis is performed based on logs and system screenshots generated during the cyclic fuzzing process, including:

[0029] Convert system screenshots into text descriptions, and then merge the text records with log records and input them into the large model;

[0030] After the large model is output, CWE vulnerability type identification and CVE vulnerability number identification are performed. The vulnerability is initially classified by matching the HarmonyOS vulnerability database, and the vulnerability category is output after manual confirmation.

[0031] On the other hand, to achieve the above objectives, the present invention also provides a multi-agent collaborative HarmonyOS API fuzz testing device, comprising:

[0032] Preprocessing agent: Used to preprocess HarmonyOS development documentation;

[0033] The driver-generated intelligent agent is used to build a large model of prompt word input using preprocessed documents, optimize prompt word templates by combining reinforcement learning, integrate Android sample code, generate corresponding driver code, and use ArkCompiler to perform syntax detection on the generated driver code to generate a high-quality seed library.

[0034] Loop testing module: used to perform loop fuzz testing using the seed library, including seed selection, mutation, compilation and execution, log collection and system screenshot;

[0035] Vulnerability Analysis Agent: Used for automated vulnerability analysis and detection based on logs and system screenshots generated during cyclic fuzzing.

[0036] A multi-agent collaborative HarmonyOS API fuzzing system includes a memory and a processor. The memory stores a computer program that is run by the processor. When the computer program is run by the processor, it executes a multi-agent collaborative HarmonyOS API fuzzing method.

[0037] A computer-readable storage medium storing a computer program that, when executed, implements a HarmonyOS API fuzzing method for multi-agent collaboration.

[0038] Compared with the prior art, the present invention has the following advantages and technical effects:

[0039] (1) Significantly improve the usability of driver generation: By introducing Android sample code and reinforcement learning to optimize prompt word templates, HMFuzzer effectively makes up for the lack of HarmonyOS API documentation and insufficient examples, and significantly improves the generation quality of test drivers;

[0040] (2) Multimodal information fusion capability: This invention innovatively transforms multimodal information such as flowcharts and life cycle diagrams in documents into text descriptions. Combined with the semantic understanding capability of large models, it solves the problem that traditional fuzzy testing tools cannot utilize visual information and improves the completeness and accuracy of test scenario construction.

[0041] (3) Security testing automation and practicality: HMFuzzer has achieved an end-to-end automated process from document preprocessing, driver generation, fuzzing to vulnerability analysis. It is compatible with real HarmonyOS physical devices and provides the first large-scale fuzzing solution that can be implemented in the HarmonyOS ecosystem. It has significant security protection and industrial application value. Attached Figure Description

[0042] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0043] Figure 1 This is a flowchart of a multi-agent collaborative HarmonyOS API fuzzing method according to an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of multimodal data conversion according to an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram illustrating the extraction of the permission configuration file according to an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram illustrating a use case of notifyCharacteristicChanged in Android according to an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of the driver generation process in an embodiment of the present invention;

[0048] Figure 6 This is a schematic diagram illustrating the recall of Android example code in an embodiment of the present invention;

[0049] Figure 7 This is a flowchart of the cyclic fuzzy testing process according to an embodiment of the present invention;

[0050] Figure 8 This is a flowchart illustrating the vulnerability analysis process according to an embodiment of the present invention.

[0051] Figure 9 This is a schematic diagram illustrating the number of available drivers generated according to an embodiment of the present invention;

[0052] Figure 10 This is a schematic diagram illustrating the number of driver repair cycles in an embodiment of the present invention;

[0053] Figure 11 This is a schematic diagram illustrating the number of available drivers generated after incorporating an Android example into an embodiment of the present invention.

[0054] Figure 12 This is a schematic diagram illustrating the number of available drivers generated after incorporating reinforcement learning in an embodiment of the present invention.

[0055] Figure 13 This is a schematic diagram illustrating the number of available drivers generated after incorporating multimodal data analysis in an embodiment of the present invention. Detailed Implementation

[0056] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0057] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0058] This embodiment proposes a fuzzing method for the HarmonyOS API, with the following main contributions: A multi-agent fuzzing framework is proposed to discover vulnerabilities in the HarmonyOS API. This framework comprises three collaborative agents (a preprocessing agent, a driver generation agent, and a vulnerability analysis agent) to identify vulnerabilities with minimal human intervention.

[0059] To increase the number of generated usable drivers, the similarities in naming, functionality, and workflow between the HarmonyOS API and the Android API are leveraged, and Android sample code is embedded as a knowledge base to compensate for the lack of HarmonyOS API examples. In the preprocessing agent, a multimodal large model is used to convert the API workflow graph (including graphs stored as JPGs) into corresponding text representations. Furthermore, a reinforcement learning scheme is employed to dynamically select cue templates to improve the generated fuzzy drivers. Finally, the vulnerability analysis agent takes test screenshots containing crash information, logs, and a vulnerability knowledge base as input and utilizes an LLM with chained reasoning to detect vulnerabilities.

[0060] This embodiment also developed a fuzzing tool called HMFuzzer and conducted a series of experiments on the HarmonyOS API. Experimental results show that our tool can not only effectively generate drivers to identify vulnerabilities in the HarmonyOS API, but also detect vulnerabilities in non-HarmonyOS targets.

[0061] A multi-agent collaborative fuzzing method for the HarmonyOS API, such as Figure 1 ,include:

[0062] Preprocess the HarmonyOS development documentation;

[0063] We construct a large model for prompt word input using preprocessed documents, optimize prompt word templates using reinforcement learning, and integrate Android sample code to generate corresponding driver code.

[0064] The generated driver code undergoes syntax checking to produce a high-quality seed library;

[0065] Cyclic fuzz testing was performed using the seed library, including seed selection, mutation, compilation and execution, log collection, and system screenshots.

[0066] Vulnerability analysis was performed based on logs and system screenshots generated during the cyclic fuzzing process.

[0067] Specifically, this embodiment uses Android sample code to make up for the lack of HarmonyOS API documentation, introduces multimodal data processing technology to convert visual information such as flowcharts in the documentation into text descriptions, and uses reinforcement learning to optimize the large model prompt word template, thereby generating high-quality, executable test drivers.

[0068] Furthermore, the HarmonyOS development documentation is preprocessed, including: document filtering, multimodal data transformation, and permission data extraction;

[0069] The document filtering process is used to filter out irrelevant grammatical documents and retain interface documents related to HarmonyOS application development; the multimodal data conversion is used to convert multimodal visual information in the documents into text data; and the permission data extraction is used to generate project permission configuration files.

[0070] Specifically, document filtering is performed: the system only tests interfaces related to HarmonyOS application development, so it is necessary to filter interface documents with other syntaxes. The filtering algorithm is shown in Table 1.

[0071] Table 1

[0072]

[0073] Input P is the set of all document paths, and Ph is the set of HarmonyOS mobile application development document paths.

[0074] Multimodal data transformation: Converting image data in a document into text descriptions, and then inserting the text descriptions into the source document.

[0075] The conversion method is as follows: extract each image from each document, use the image as input to the multimodal large model, output the text description in the image, and insert the output result at the end of the original document, such as... Figure 2 As shown.

[0076] Permission data extraction: To avoid false positives due to insufficient permissions during the cyclic fuzzing phase, all permission data is extracted and compiled into a permission document before fuzzing. The algorithm is shown in Table 2.

[0077] Table 2

[0078]

[0079]

[0080] The algorithm takes AP as input, which is a set of permission document paths. It iterates through AP and parses out the permission name auth_name, permission description auth_des, permission level auth_level, authorization mode auth_mode, and valid version auth_valid for each document. Finally, it outputs the permission configuration file AF. Figure 3 The left side shows the original document, and the right side shows the permission configuration file after being transformed by the algorithm.

[0081] Further, generating the corresponding driver code includes:

[0082] The optimal template is selected from a predefined prompt word template library based on reinforcement learning;

[0083] Recall Android sample code that matches the semantics of the current function and integrate it into the prompt word input model to generate the corresponding driver code.

[0084] Specifically, the preprocessed documents and Android sample code are used as part of the prompt words. Reinforcement learning techniques are used to optimize the prompt words, and the optimized prompt words are used as input to the large model to generate the driving code.

[0085] The quality of prompt words directly affects the relevance and accuracy of the output content of the large language model through factors such as clarity, specificity, accuracy, and style requirements. In the driver code generation phase, this embodiment carefully designed 10 prompt word templates and used reinforcement learning techniques to select suitable templates to increase the number of executable driver code generated. However, in some functional units, the lack of more specific use cases—for example, in the Bluetooth development documentation, although the notifyCharacteristicChanged() API (which actively notifies connected client devices when server-side feature values ​​change) is described, no usage examples are provided—causes the large language model to tend to generate drivers that include API use cases. On the other hand, due to the long history of the Android system and its well-developed open-source ecosystem, a large number of use cases have been accumulated for reference.

[0086] For example, Android also has similar APIs for reference, such as the notifyCharacteristicChanged() API for HarmonyOS. Figure 4 Therefore, when constructing prompt words, semantic similarity is used to perform a recall operation in the Android sample library. The recalled Android sample code is combined with preprocessed documents and incorporated into the selected prompt word template. This is then input into a large language model to generate driving code. The compiled feedback information can be used to guide reinforcement learning in selecting prompt word templates.

[0087] Furthermore, during the generation of the corresponding driver code, the agent selects the optimal template, the state data comes from the compilation feedback data, and the reward function is selected based on the compilation success rate, error repair difficulty, and execution efficiency optimization prompts.

[0088] Specifically, in driver code generation, constructing appropriate prompt words is crucial for improving the quality and executability of generated drivers. However, the construction and selection of prompt word templates involve certain subjective factors, and an unreasonable selection of prompt word templates may lead to low model output quality. Therefore, a batch of driver generation prompt word templates are constructed according to current mainstream prompt word design principles. Reinforcement learning is then used to select appropriate prompt word templates to increase the number of executable output drivers, thereby increasing the diversity of test cases.

[0089] The following are the relevant designs:

[0090] (1) Agent: The agent is responsible for selecting the optimal template from a predefined prompt word template library.

[0091] (2) Environment: The environment includes large language models, compiler feedback and code generation tasks.

[0092] (3) State: In reinforcement learning, state data mainly comes from compilation feedback, which is divided into two categories: compilation success (including warnings and complete success) and compilation failure (covering syntax errors, semantic errors, linking errors and preprocessor errors).

[0093] The state vector is defined as follows:

[0094] S = [s flag ,s warn ,s ge ,s se ,s le ,s pe ],

[0095] Among them, s flag These are compilation flags: 0: compilation failed; 1: compilation succeeded but with warnings; 2: compilation succeeded completely.warn Represents the number of warnings. s ge Indicates the number of syntax errors. s se Indicates the number of semantic errors. s le Indicates the number of broken links. s pe This represents the number of preprocessor errors.

[0096] Furthermore, different error types vary in the ease of driver repair, which affects the driver generation speed and further restricts the efficiency of the entire fuzzing process; therefore, this must also be taken into account. Error types are listed in Table 3.

[0097] Table 3

[0098]

[0099] (4) Action: The action is to select a prompt template from the prompt word library.

[0100] (5) Reward: The reward is based on metrics such as code compilation success rate and execution efficiency. The reward function R is:

[0101]

[0102] In the formula, R represents the reward value, and E j ={s warn ,s ge ,s se ,s le ,s pe} represents the set of errors and warnings (s) warn Represents the number of warnings, s ge Indicates the number of syntax errors, s se Indicates the number of semantic errors, s le Indicates the number of broken links, s pe (representing the number of preprocessor errors), T represents the generation time of the current driver, s flag Indicates a compilation flag, ω flag For the compilation flag weight, ω j For the weights of the relevant error types, ω T To drive the generation of time weights, T max The maximum time required to drive generation. The overall flowchart is as follows: Figure 5 As shown.

[0103] To address the issue that some HarmonyOS APIs lack test cases, preventing large models from generating correct function calls and ensuring sufficient testing of related functions, this embodiment introduces the Android sample library (Android APIs and HarmonyOS APIs share many design similarities). This approach leverages the Android sample library to provide more comprehensive test cases for large models, thereby generating more comprehensive test-driven implementations.

[0104] like Figure 6 The Android sample code retrieval method is mainly divided into two modules: sample code storage and sample code retrieval. Sample code storage is mainly responsible for structuring and vectorizing the Java code in the Android project. Sample code retrieval consists of three stages: stage 1 extracts the title and API description from the source document; stage 2 uses an embedding model for vectorization; and stage 3 uses a cosine similarity matching algorithm to find relevant Android code snippets from the vector database.

[0105] Furthermore, cyclic fuzz testing is performed using the seed library, including:

[0106] Step 1: Initialize the energy value of the driver code in the seed library;

[0107] Step 2: Select seeds based on the energy values, and perform mutation operations on the selected seeds using a large model;

[0108] Step 3: Compile and run the mutated code, and collect logs and system screenshots simultaneously;

[0109] Step 4: Update the seed energy value based on the log analysis results, and iterate through steps 2-4.

[0110] Specifically, the overall flowchart of the cyclic fuzzing test is as follows: Figure 7 As shown, firstly, all generated drivers are stored in a seed library after undergoing random initialization energy processing. Then, seeds are selected based on energy levels, and a large model is used to mutate the selected seeds. After mutation, the project is built, compiled, and run. During this process, logs are collected and analyzed simultaneously, and system screenshots are taken (to assist in vulnerability analysis). Finally, energy calculations are performed based on the log analysis results.

[0111] Seed selection can be performed using energy. Seed energy calculation primarily considers the presence of crashes in the collected screenshots, the number of log lines (log line count reflects coverage to some extent), and the number of logs at different severity levels (different log severity levels correspond to different test result severity levels, ranked from low to high as debug, info, warn, error, fatal). Logs at the warn level and above are more valuable for energy calculation; therefore, only logs at the warn level and above are considered. The energy calculation formula is as follows:

[0112]

[0113] In the formula, N log Represents the number of log lines; NL is the vector of log lines at the warn level and above, NL = {nl w ,nl e ,nl f}, nl w Indicates the number of warning log lines, nl e Indicates the number of warning log lines, nl f Indicates the number of fatal error log lines, NL i S is a sub-element of vector NL. flag This indicates whether a crash issue exists in the system screenshot (1: indicates presence, 0: indicates absence); ω log The weight of the number of log lines, ω i ω represents the weight of the number of rows for each log type. flag To test crash weights.

[0114] Furthermore, vulnerability analysis is performed based on logs and system screenshots generated during the cyclic fuzzing process, including:

[0115] Convert system screenshots into text descriptions, and then merge the text records with log records and input them into the large model;

[0116] After the large model is output, CWE vulnerability type identification and CVE vulnerability number identification are performed. The vulnerability is initially classified by matching the HarmonyOS vulnerability database, and the vulnerability category is output after manual confirmation.

[0117] Specifically, such as Figure 8 As shown, during the cyclic fuzzing phase, logs and system screenshots are collected. The system screenshots need to be converted into text descriptions to be used as prompts for the large model along with the log records. After the large model outputs, it first identifies the CWE vulnerability type, and then identifies the CVE vulnerability. The vulnerability identification process matches the vulnerability database of HarmonyOS 5.0 up to the present to determine the vulnerability category. Finally, it is manually confirmed.

[0118] This embodiment also provides a multi-agent collaborative HarmonyOS API fuzz testing device, including:

[0119] Preprocessing module: Used to preprocess ArkTS documents;

[0120] Driver generation module: Used to build a large model of prompt word input using preprocessed documents, optimize prompt word templates by combining reinforcement learning, and integrate Android sample code to generate corresponding driver code;

[0121] Syntax detection module: Used to perform syntax checks on the generated driver code and generate a high-quality seed library;

[0122] Loop testing module: used to perform loop fuzz testing using the seed library, including seed selection, mutation, compilation and execution, log collection and system screenshot;

[0123] Vulnerability Analysis Module: Used for vulnerability analysis based on logs and system screenshots generated during cyclic fuzzing.

[0124] Specifically, the preprocessing module includes a document filter unit, a multimodal data conversion unit, and a permission data extraction unit. The document filter retains the interface documents related to HarmonyOS application development; the multimodal data conversion unit converts the multimodal data in the HarmonyOS application development documents into text data; to avoid fuzzing failure due to insufficient permissions during the fuzzing phase, a permission data extraction unit is introduced, which mainly generates the project permission configuration file based on the permission document.

[0125] The driver generation module meticulously constructs a large-scale input model using preprocessed documents and combines it with reinforcement learning to generate corresponding driver code. The syntax checking module is responsible for syntax checking of the driver code, ensuring its executability and generating a high-quality seed library. The loop testing module uses the generated seed library, in conjunction with the syntax checking and driver generation modules, to perform mutation and fuzzing tests. The vulnerability analysis module uses logs and system screenshots generated during fuzzing to assist manual vulnerability analysis.

[0126] A multi-agent collaborative HarmonyOS API fuzzing system includes a memory and a processor. The memory stores a computer program that is run by the processor. When the computer program is run by the processor, it executes a multi-agent collaborative HarmonyOS API fuzzing method.

[0127] A computer-readable storage medium storing a computer program that, when executed, implements a HarmonyOS API fuzzing method for multi-agent collaboration.

[0128] To more clearly illustrate the technical solution of the present invention, specific embodiments are provided below for description:

[0129] To generate the HarmonyOS driver, we crawled the official HarmonyOS Arkts documentation (1413 documents in total) and Android sample code from GitHub (231 Android sample codes) as data sources. Considering the computational cost, we used Python to call HTTP interfaces to access the large models deepseek, qwen, chatgpt, and qwenVL. While deepseek, qwen, and chatgpt were primarily used for driver code generation, deepseek was ultimately chosen as the large model for driver generation (related comparative data is shown in Figure *). We used qwenVL for multimodal data processing and analysis. The HarmonyOS command-line compilation and build tools were used for compilation and execution. The compiled results were run on a Huawei nova12 Ultra physical machine. The experiment was conducted in three rounds, each lasting 360 hours.

[0130] like Figure 9 and Figure 10 The figure shows the number of executable drivers generated by Deepseek, Qwen, and ChartGPT based on the original interface documents (considering the cost of GPT interface calls, only the first 200 documents were tested, and all documents were used in the fuzzing test loop phase). Deepseek was chosen for driver generation because its driver generation and repair capabilities are better than the other two large models.

[0131] This embodiment evaluates the vulnerability discovery efficiency of HMFuzzer. Table 4 shows the vulnerability discovery capabilities of HMFuzzer, Fuzz4All, and TitanFuzz on the HarmonyOS 5.0 system.

[0132] Table 4

[0133]

[0134]

[0135] HMFuzzer, Fuzz4All, and TitanFuzz all generate a large number of false positives. After manual analysis and inspection, HMFuzzer was found to have a known vulnerability (CVE-2024-58115, SVG parsing buffer). The other two fuzzing tools did not find any new vulnerabilities. This may be because HMFuzzer introduces sample code from Android, which contains a wealth of SVG code examples that help generate related drivers.

[0136] To verify the improvement in the number of availability drivers generated by adding the Android sample code, the number of availability drivers generated before and after the addition was compared, such as... Figure 11 As shown, the number of usability drivers generated increased by 16% after adding the Android sample code, indicating that the Android sample code can make up for the deficiencies of some HarmonyOS API sample code.

[0137] To verify the improvement rate of reinforcement learning on the number of usability-driven generation, reinforcement learning was used both with the original documentation and with the addition of Android sample code, as follows: Figure 12 The figure shows the number of usability-driven code generated in these two scenarios. Using reinforcement learning in the original document improved usability-driven code generation by 17%, and adding Android sample code improved it by 24%. This demonstrates that reinforcement learning can further improve the generation rate of usability-driven code.

[0138] To verify the improvement rate of the number of usability drivers generated by adding multimodal analysis, the number of usability drivers generated before and after the addition was compared, such as... Figure 13 As shown, the number of availability-driven APIs generated increased by 4% after introducing multimodal data preprocessing of the original documents, indicating that multimodal data has a certain guiding effect on the generation of availability-driven APIs for some APIs.

[0139] This embodiment discloses a multi-agent collaborative HarmonyOS API fuzzing framework, HMFuzzer. HMFuzzer integrates five modules—preprocessing, driver generation, syntax checking, loop testing, and vulnerability analysis—through multi-agent collaboration, systematically improving testing efficiency and vulnerability discovery capabilities. Core innovations include: utilizing Android sample code to compensate for missing HarmonyOS API documentation; introducing multimodal data processing technology to convert visual information such as flowcharts in the documentation into text descriptions; and employing reinforcement learning to optimize large-scale model prompt word templates, thereby generating high-quality, executable test drivers. Experiments were conducted on a real HarmonyOS 5.0 device. HMFuzzer successfully discovered one confirmed vulnerability (CVE-2024-58115), while the comparison tools Fuzz4All and TitanFuzz failed to find any new vulnerabilities. Ablation experiments show that Android sample code, reinforcement learning, and multimodal processing improved driver usability by 16%, 24%, and 4%, respectively, providing the first multi-agent-based fuzzing solution for HarmonyOS system security testing.

[0140] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-agent collaborative fuzzing method for the HarmonyOS API, characterized in that, include: Preprocess the HarmonyOS development documentation; We construct a large model for prompt word input using preprocessed documents, optimize prompt word templates using reinforcement learning, and integrate Android sample code to generate corresponding driver code. Perform syntax checking on the generated driver code to produce a high-quality seed library; The seed library was used to perform cyclic fuzz testing, including seed selection, mutation, compilation and execution, log collection, and system screenshots. Vulnerability analysis was performed based on logs and system screenshots generated during the cyclic fuzzing process.

2. The HarmonyOS API fuzzing method for multi-agent collaboration according to claim 1, characterized in that, The HarmonyOS development documentation is preprocessed, including: document filtering, multimodal data conversion, and permission data extraction. The document filtering process is used to filter out irrelevant grammatical documents and retain interface documents related to HarmonyOS application development; the multimodal data conversion is used to convert multimodal visual information in the documents into text data; and the permission data extraction is used to generate project permission configuration files.

3. The HarmonyOS API fuzz testing method for multi-agent collaboration according to claim 2, characterized in that, The multimodal data conversion includes: Extract each image from the document, input a multimodal large model to generate a text description, and insert the text description at the end of the original document.

4. The HarmonyOS API fuzz testing method for multi-agent collaboration according to claim 1, characterized in that, Generating the corresponding driver code includes: The optimal template is selected from a predefined prompt word template library based on reinforcement learning; Recall Android sample code that matches the semantics of the current function and integrate it into the prompt word input model to generate the corresponding driver code.

5. The HarmonyOS API fuzzing method for multi-agent collaboration according to claim 4, characterized in that, During the generation of the corresponding driver code, the agent selects the optimal template, the state data comes from the compilation feedback data, and the reward function is selected based on the compilation success rate, error repair difficulty, and execution efficiency optimization prompts.

6. The HarmonyOS API fuzzing method for multi-agent collaboration according to claim 1, characterized in that, Performing cyclic fuzz testing using the aforementioned seed library includes: Step 1: Initialize the energy value of the driver code in the seed library; Step 2: Select seeds based on the energy values, and perform mutation operations on the selected seeds using a large model; Step 3: Compile and run the mutated code, and collect logs and system screenshots simultaneously; Step 4: Update the seed energy value based on the log analysis results, and iterate through steps 2-4.

7. The HarmonyOS API fuzz testing method for multi-agent collaboration according to claim 1, characterized in that, Vulnerability analysis was conducted based on logs and system screenshots generated during the cyclic fuzzing process, including: Convert system screenshots into text descriptions, and then merge the text records with log records and input them into the large model; After the large model is output, CWE vulnerability type identification and CVE vulnerability number identification are performed. The vulnerability is initially classified by matching the HarmonyOS vulnerability database, and the vulnerability category is output after manual confirmation.

8. A multi-agent collaborative HarmonyOS API fuzz testing device, characterized in that, include: Preprocessing agent: Used to preprocess HarmonyOS development documentation; The driver-generated intelligent agent is used to build a large model of prompt word input using preprocessed documents, optimize prompt word templates by combining reinforcement learning, integrate Android sample code, generate corresponding driver code, and use ArkCompiler to perform syntax detection on the generated driver code to generate a high-quality seed library. Loop testing module: used to perform loop fuzz testing using the seed library, including seed selection, mutation, compilation and execution, log collection and system screenshot; Vulnerability Analysis Agent: Used for automated vulnerability analysis and detection based on logs and system screenshots generated during cyclic fuzzing.

9. A multi-agent collaborative HarmonyOS API fuzz testing system, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that is executed by the processor, and the computer program, when executed by the processor, performs the HarmonyOS API fuzz testing method for multi-agent cooperation as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed, implements the HarmonyOS API fuzzing method for multi-agent collaboration as described in any one of claims 1-7.

Images