Driving software automatic testing method and platform based on AI intelligent agent

Abstract

The invention is suitable for the technical field of driver software testing, and provides a driver software automatic testing method and platform based on an AI agent. According to the method, full-automatic operation of test management, equipment management and a test process of the driver software is realized, a complex environment of a bottom driver software test is flexibly coped with through an artificial intelligence technology, manpower required by the test is reduced, and the test quality and the test efficiency are improved. Meanwhile, by defining the test stage, the uncertainty degree of the artificial intelligence technology in the aspect of content generation is reduced, and the test stability is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of driver software testing, and in particular relates to an AI-based automated driver software testing method and platform. Background Art

[0002] Unlike application software testing, driver software needs to adapt to different types of hardware characteristics and protocols, has variable software and hardware collaboration issues, large amounts of code and complex logic, and is characterized by high testing difficulty, insufficient test coverage, and cumbersome test preparation. In this case, R&D personnel are often required to deeply participate in the testing process, and the degree of automation and testing efficiency are both low.

[0003] Expanding to the field of software testing, currently published patents include methods for automated testing of task flows in machine learning platforms, real-time automated testing of SDKs, automated testing of embedded software, and automated testing of embedded code in Git management tools. However, these methods generally focus on improving limited parts of the automated testing platform, rather than platform-level design improvements. Furthermore, they fail to integrate cutting-edge AI technologies into platform-oriented architecture design and application implementation. Even automated testing platforms for embedded software fail to address coordination with hardware.

[0004] With the rapid development of artificial intelligence technology in recent years, there are more and more scenarios in which various large models are deployed and applied within enterprises. Therefore, applying artificial intelligence technology and large model capabilities to drive testing and providing a set of intelligent solutions can improve testing capabilities and efficiency. Summary of the Invention

[0005] In view of the above problems, the purpose of the present invention is to provide an AI-based automated testing method and platform for driver software, aiming to solve the technical problems of low test quality and test efficiency of existing testing schemes.

[0006] The present invention adopts the following technical solutions:

[0007] On the one hand, the AI agent-based driver software automated testing method includes the following steps:

[0008] Step S1: The test execution agent receives and parses the test input file sent by the user, then obtains the test case set data from the test database and generates a set of test tasks;

[0009] Step S2: The device management agent arranges the test task execution order based on the status of the target test machine corresponding to each test task, calls the AI tool set according to the current test task requirements, pulls the test artifacts from the artifact library, and finally sends the test artifacts and test instructions to the test target machine;

[0010] Step S3: The test target machine executes the test action according to the test instruction, and finally returns the test log and test result data;

[0011] Step S4: The device management agent collects the test log and test result data and sends them to the test execution agent;

[0012] Step S5: Every time the test execution agent receives the test result data of a test task, it calls the AI tool set to determine whether the test passes. After all tests are completed, it calls the AI tool set to generate a test report.

[0013] Furthermore, the automated testing method further includes:

[0014] Step S6: The test execution agent uploads the test log, test report and all test result data to the artifact library, and returns the test report to the user.

[0015] On the other hand, the AI-agent-based driver software automated testing platform includes: a test execution agent, a device management agent, a hardware resource pool composed of a target tester, a test database, a product library, and an AI tool set, wherein the test execution agent and the device management agent are implemented in the form of AI agents;

[0016] The test execution agent is used to receive and parse the test input files sent by the user, then obtain the test case set data from the test database and generate a set of test tasks. It is also used to call the AI tool set to determine whether the test passes after receiving the test result data of each test task, and to call the AI tool set to generate a test report after all tests are completed. It is also used to upload the test log, test report and all test result data to the artifact library and return the test report to the user.

[0017] The device management agent is responsible for arranging the test task execution sequence based on the status of the target test machine corresponding to each test task, calling the AI tool set according to the current test task requirements, pulling test artifacts from the artifact library, and finally sending the test artifacts and test instructions to the test target machine; and is also responsible for collecting test logs and test result data and sending them to the test execution agent;

[0018] The test target machine is used to execute test actions according to test instructions and finally return test logs and test result data.

[0019] The beneficial effects of the present invention are as follows: the present invention is based on a test execution agent and a device management agent, and the two AI agents work together to realize the automatic generation of a test plan, test task decomposition, acquisition of test data, scheduling of hardware equipment, execution of test tasks, acquisition of test logs, judgment of test results and generation of test reports after the user inputs the test goal, thereby realizing full automation of the underlying driver software testing process. The present invention flexibly responds to the complex environment of the underlying driver software testing through artificial intelligence technology, reduces the manpower required for testing, and improves test quality and efficiency. At the same time, by defining the test phase, the uncertainty of artificial intelligence technology in content generation is reduced, and the stability of the test is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of an AI-agent-based automated testing platform for driver software provided by an embodiment of the present invention;

[0021] Figure 2 This is a flowchart of the AI agent-based automated testing method for driver software provided by an embodiment of the present invention;

[0022] Figure 3 It is a specific interaction diagram of the automated testing method;

[0023] Figure 4 is the state transition diagram of the test execution agent;

[0024] Figure 5 It is the state transition diagram of the device management agent. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0026] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0027] Example 1:

[0028] like Figure 1 As shown, the AI agent-based driver software automated testing platform provided in this embodiment includes a test execution agent, a device management agent, a hardware resource pool composed of target testers, a test database, a product library, and an AI tool set. The test execution agent and the device management agent are implemented in the form of AI agents. Specifically:

[0029] The test execution agent is used to receive and parse the test input files sent by the user, then obtain the test case set data from the test database and generate a set of test tasks. It is also used to call the AI tool set to determine whether the test passes after receiving the test result data of each test task, and to call the AI tool set to generate a test report after all tests are completed. It is also used to upload the test log, test report and all test result data to the artifact library and return the test report to the user.

[0030] The device management agent is responsible for arranging the test task execution sequence based on the status of the target test machine corresponding to each test task, calling the AI tool set according to the current test task requirements, pulling test artifacts from the artifact library, and finally sending the test artifacts and test instructions to the test target machine; and is also responsible for collecting test logs and test result data and sending them to the test execution agent;

[0031] The test target machine is used to execute test actions according to test instructions and finally return test logs and test result data.

[0032] As can be seen from the various functions of the aforementioned platform, the test execution agent in this embodiment is implemented as an AI agent. It parses the task requirements based on the test input file (in a structured text format such as XML or JSON) sent by the user, then obtains data from the test database to generate a set of test tasks in stages. It then interacts with the device management agent and calls tools in the AI toolset to perform test judgments and generate test reports. The test phases are divided into: obtaining the test input file, executing the test task, judging the test results, and generating the test report. In each stage, the agent generates the test tasks and test instructions for specific test cases.

[0033] The device management agent is also implemented in the form of an AI agent, which receives and parses the test tasks sent by the test execution agent, finds the test target machine corresponding to the test task, and arranges the execution order of the test task according to the test target machine status and load, and calls the AI toolset as needed to pull products from the product library. The device management agent includes a device status monitoring module, a communication module, and a data transmission module, which respectively implement three functions: device status monitoring function (obtaining the test target machine status), communication function (sending test instructions to the test target machine and recovering the test log), and data transmission function (sending test task data to the test target machine and recovering the test result data). After the test execution is completed, the device management agent returns the test log and test result data to the test execution agent.

[0034] The AI toolset is a set of AI tools invoked by the test execution agent and the device management agent, implemented through a unified interface. These tools include artifact retrieval tools, image test result judgment tools, log test result judgment tools, and test report generation tools. These tools are customized based on large model capabilities and feature a unified interface and return format, facilitating platform invocation, functional adjustment, and expansion.

[0035] The test database stores test case information and returns the test case set data information after receiving the query instruction sent by the test execution agent.

[0036] The product library stores the products to be tested and other products of the procedures required for the test, and stores the test records and test reports after the test is completed.

[0037] It should be noted that the two AI agents of the automated testing platform of this embodiment are implemented based on a general large language model deployed locally within the enterprise. Specifically, they access the enterprise knowledge base as their background knowledge source, and then design special prompts (Prompts) for the driving test task content and execution process and perform fine-tuning based on the Prompts.

[0038] The automated testing platform provided by the embodiments of the present invention fully automates driver software test management, device management, and test processes. Leveraging artificial intelligence technology, it flexibly addresses the complex environment of underlying driver software testing, reducing the manpower required for testing and improving both test quality and efficiency. Furthermore, by defining test phases, it reduces the uncertainty of AI content generation and improves test stability.

[0039] Example 1:

[0040] like Figure 2 、 3 As shown, this embodiment provides an AI agent-based driver software automated testing method, which is implemented based on the automated testing platform shown in Example 1. The automated testing method includes the following steps:

[0041] Step S1: The test execution agent receives and parses the test input file sent by the user, then obtains the test case set data from the test database and generates a set of test tasks.

[0042] The user sends the test input file to the platform. The test input file is in a structured format such as JSON / XML and contains information such as the test purpose, test module, test duration, target platform, etc. The specific process of step S1 is as follows:

[0043] S11. The test execution agent receives and parses the test input file sent by the user, and generates a database query instruction based on its knowledge of the test database. The test database receives the query instruction and returns a set of test case data.

[0044] S12. The test execution agent receives the test case set data.

[0045] S13. The test execution agent parses and generates specific test tasks based on each set of test case data in the test case set data to obtain a set of test tasks.

[0046] Step S2: The device management agent arranges the execution order of the test tasks according to the status of the target test machine corresponding to each test task, calls the AI toolset according to the current test task requirements, pulls the test products from the product library, and finally sends the test products and test instructions to the test target machine.

[0047] The specific process of this step is as follows:

[0048] S21. The device management agent receives a set of test tasks, parses the test target machine information corresponding to each test task, and obtains the status of the test target machine through the device status monitoring module;

[0049] S22, arranging the test task execution order based on the status of each test target machine;

[0050] S23. In accordance with the execution order, the device management agent calls the artifact pulling tool of the AI tool set according to the current test task requirements and pulls the test artifacts from the artifact library;

[0051] S24 , sending the test product to the test target machine through the data transmission module, and sending the test instruction to the test target machine through the communication module.

[0052] In this step, each test task has a corresponding test target machine, and each test target machine has different states, such as load status. Upon receiving a set of test tasks, data parsing is required to determine, for example, how many test tasks are included and what the test target machine is for each test task. The device status monitoring module then obtains the state of the test target machine and, based on the actual situation, determines the order in which the test tasks will be executed, effectively orchestrating the test process. Following the specific test execution sequence, test artifacts are sequentially pulled from the artifact library for each test task, and corresponding test instructions are generated. The test tasks and test instructions are then sent to the corresponding test target machines.

[0053] Step S3: The test target machine executes the test action according to the test instruction, and finally returns the test log and test result data.

[0054] The test target machine executes the test action according to the test instruction, and returns the test log to the test execution agent through the communication module, and returns the test result data to the test execution agent through the data transmission module.

[0055] Step S4: The device management agent collects the test log and test result data and sends them to the test execution agent.

[0056] The device management agent recycles the test log and test result data. After each test task is completed, the device management agent returns the recycle test log and test result data to the test execution agent.

[0057] Step S5: Every time the test execution agent receives the test result data of a test task, it calls the AI tool set to determine whether the test passes. After all tests are completed, it calls the AI tool set to generate a test report.

[0058] Each time the test execution agent receives the test log and test result data of a test task, it calls the image test result judgment tool and log test result judgment tool of the AI tool set to determine whether the test has passed. After completing all tests, it calls the test report generation tool of the AI tool set to generate a test report.

[0059] Step S6: The test execution agent uploads the test log, test report and all test result data to the artifact library, and returns the test report to the user.

[0060] The following example shows a specific application example of the automated testing method. The specific process is as follows:

[0061] 1) User sends test input file (for example):

[0062]

[0063]

[0064] 2) After the test execution agent parses the JSON file, it determines the required test cases and sends query SQL instructions to the test database;

[0065] 3) The test database sends a set of test case data that meets the requirements (for example):

[0066] Test case number Module Test case type Test case name Test case description ... 1 3D module Functional testing Glmark2 ... ... 2 Typical Applications Functional testing CTS ... ... ... ... ... ... ... ...

[0067] 4) The test execution agent generates test task information based on the test case data and sends the test task information to the device management agent;

[0068] 5) The device management agent classifies and sorts the test tasks by type (device status query task, product pull task, data transmission task, information communication task) and target platform, and schedules its submodules to complete the test tasks;

[0069] 6) After the device management agent completes a single test task (such as glmark2), it sends its test log and display information to the test execution agent. The test execution agent calls the image test result judgment tool and the log test result judgment tool to generate the test result of this test task;

[0070] 7) After all test tasks are completed, the test execution agent calls the test report generation tool to generate a test report;

[0071] 8) The test execution agent uploads the test report and test result data to the artifact library;

[0072] 9) The user views the test report.

[0073] Through the method of the embodiment of the present invention, it is possible to process test tasks according to the specific test content input by the user, reduce the need for manual activities due to the complex test environment unique to driver software testing, and improve test efficiency; and by processing test task data management, test process follow-up, test status records and test report generation and storage, reduce the need for manpower in test process supervision activities; at the same time, the completeness of its status records can also improve test quality and reproducibility. The automated testing platform of the present invention also provides a set of AI tools based on AI capabilities, and has a standardized calling interface and return format, so that the platform functions can be easily expanded.

[0074] It is also important to note that this invention specifically designs two AI agents for testing and the interaction between them, enabling intelligent execution of automated testing, improving testing efficiency and reducing manpower requirements. The design and interaction of the two agents are described in detail below:

[0075] 1. Test Execution Agent

[0076] The test execution agent has five states, namely processing state A (Processing1), waiting state (Waiting), processing state B (Processing2), completed state (Done) and idle state (Idle). The state jump process is as follows Figure 4 shown.

[0077] The test execution agent is initially in the idle state and switches to the processing state A after receiving the test input file sent by the user;

[0078] In processing state A, a special prompt word is designed to make the test execution agent perform the following steps:

[0079] a. Parse the test input file;

[0080] b. Generate database query instructions;

[0081] c. Send database query instructions;

[0082] d. Receive test case set data;

[0083] e. Parse to obtain a set of test tasks;

[0084] f. Send test tasks to the device management agent;

[0085] g. And switch to waiting state.

[0086] In the waiting state, if the test log returned by the device management agent is received, it will be converted to processing state B:

[0087] In processing state B, a special prompt word is designed to make the test execution agent perform the following steps:

[0088] a. Parse test logs;

[0089] b. Call the image test result judgment tool and the log test result judgment tool to determine whether the test passes;

[0090] c. Call the test report generation tool to generate a test report for this round of testing;

[0091] d. Return the test report to the user and finally convert it to the completed status.

[0092] In the completed state, when the user confirms the end of this round of testing, the state transitions from the completed state to the idle state.

[0093] 2. Test Execution Agent

[0094] The device management agent has three states: processing state C (Processing), completed state (Done) and idle state (dle); the state jump process is as follows Figure 5 shown.

[0095] The device management agent is initially in the idle state. After receiving the test task sent by the test execution agent, it switches to the processing state C.

[0096] In processing state C, a dedicated prompt word is designed to enable the device management agent to perform the following steps:

[0097] a. For a set of test tasks, parse the test target machine information corresponding to each test task, obtain the status of the test target machine, and arrange the test task execution order;

[0098] b. When each test task is executed, the artifact pulling tool is called to pull the test artifacts;

[0099] c. Send test products and test instructions to the test target machine;

[0100] d. Recover the test log and test result data and return them to the test execution agent, converting them to the completed state;

[0101] In the completed state, when the user confirms the end of this round of testing, the state transitions from the completed state to the idle state.

[0102] 3. Interaction between the Test Execution Agent and the Device Management Agent

[0103] Because two AI agents may run on different physical devices, in this invention, the two AI agents communicate using HTTP, and the exchanged data between them is designed to be structured data formats such as JSON / XML. Each AI agent maintains a message queue, including messages to be sent and metadata.

[0104] The process of the test execution agent sending information to the device management agent is as follows:

[0105] a. The test execution agent adds the test task data to the message queue and attempts to send it immediately;

[0106] b. If a successful response message (HTTP 200) is received, the data is removed from the message queue;

[0107] c. If a failure message (HTTP 503) is received, resend after the preset time next_retry_time. If it still fails after trying the maximum number of retries, discard the test task and output the test failure result.

[0108] The process of the device management agent receiving and processing the information sent by the test execution agent is as follows:

[0109] a. First, perform a status check to determine whether the current test execution agent is in a waiting state (Waiting) and whether the device management agent is in an idle state (Idle);

[0110] b. If both are true, the status is correct. The test task data is added to the message queue of the device management agent. The test task is processed according to the process and a successful response is returned.

[0111] c. Otherwise, it indicates an error status and returns an error response.

[0112] Finally, the present invention designs the test execution agent and the device management agent in the form of a state machine. The execution steps of each agent in different states and the jumps between states are all reflected in the process steps of Example 2. The main functions of the test execution agent are: parsing user input files and intelligently generating database query instructions; generating test tasks based on test case information, cooperating with the device management agent to complete test tasks, calling AI toolsets, judging test results, and generating test reports, etc. The main functions of the device management agent are: receiving test tasks, performing task parsing and classification, automatically scheduling the execution process of tasks on the hardware platform, and returning test result data. By transferring structured test task information between agents, the test execution agent and the device management agent cooperate to complete the test together. During the test execution, the agent is flexibly scheduled to deal with various test situations without manual participation, thereby improving test quality and test efficiency.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An automated testing method for driver software based on AI agent, characterized in that: The automated testing method comprises the following steps: Step S1: The test execution agent receives and parses the test input file sent by the user, then obtains the test case set data from the test database and generates a set of test tasks; Step S2: The device management agent arranges the test task execution order based on the status of the target test machine corresponding to each test task, calls the AI tool set according to the current test task requirements, pulls the test artifacts from the artifact library, and finally sends the test artifacts and test instructions to the test target machine; Step S3: The test target machine executes the test action according to the test instruction, and finally returns the test log and test result data; Step S4: The device management agent collects the test log and test result data and sends them to the test execution agent; Step S5: Every time the test execution agent receives the test result data of a test task, it calls the AI tool set to determine whether the test passes. After all tests are completed, it calls the AI tool set to generate a test report.

2. The AI agent-based automated testing method for driver software according to claim 1, wherein: The automated testing method further comprises: Step S6: The test execution agent uploads the test log, test report and all test result data to the artifact library, and returns the test report to the user.

3. The AI agent-based driver software automated testing method according to claim 1 or 2, wherein: The two AI agents, the test execution agent and the device management agent, are implemented based on a general large language model deployed locally within the enterprise. Specifically, they access the enterprise knowledge base as a source of background knowledge, and then design special prompt words for the test task content and execution process of the driver software, and perform fine-tuning based on the prompt words.

4. The AI agent-based automated testing method for driver software according to claim 3, wherein: The specific process of step S1 is as follows: S11. The test execution agent receives and parses the test input file sent by the user and generates a database query instruction based on its knowledge of the test database. The test input file is in a structured format and includes information about the test purpose, test modules, test duration, and test target machine. S12. The test execution agent receives the test case set data returned by the test database according to the query instruction; S13. The test execution agent parses and generates specific test tasks based on each set of test case data in the test case set data to obtain a set of test tasks.

5. The AI agent-based automated testing method for driver software according to claim 4, wherein: The equipment management intelligent body includes an equipment status monitoring module, a communication module and a data transmission module. The AI tool set includes a product pulling tool, an image test result judgment tool, a log test result judgment tool, and a test report generation tool. These tools are customized based on large model capabilities and have a unified interface and return format, which facilitates calling, function adjustment and expansion.

6. The AI agent-based automated testing method for driver software according to claim 5, wherein: The specific process of step S2 is as follows: S21. The device management agent receives a set of test tasks, parses the test target machine information corresponding to each test task, and obtains the status of the test target machine through the device status monitoring module; S22, arranging the test task execution order based on the status of each test target machine; S23. In accordance with the execution order, the device management agent calls the artifact pulling tool of the AI tool set according to the current test task requirements and pulls the test artifacts from the artifact library; S24, sending the test product to the test target machine through the data transmission module, and sending the test instruction to the test target machine through the communication module; In step S3, the test target machine executes the test action according to the test instruction, and returns the test log to the test execution agent through the communication module, and returns the test result data to the test execution agent through the data transmission module; In step S4, the device management agent recovers the test log and test result data. After each test task is completed, the device management agent returns the recovered test log and test result data to the test execution agent. In step S5, each time the test execution agent receives the test log and test result data of a test task, it determines whether the test passes by calling the image test result judgment tool and log test result judgment tool of the AI tool set. After completing all tests, it calls the test report generation tool of the AI tool set to generate a test report.

7. The AI agent-based automated testing method for driver software according to claim 6, wherein: The test execution agent has five states, namely processing state A, waiting state, processing state B, completion state and idle state; The test execution agent is initially in the idle state and switches to the processing state A after receiving the test input file sent by the user; In processing state A, a dedicated prompt word is designed to cause the test execution agent to perform the following steps: parse the test input file, generate database query instructions, send the database query instructions, receive the test case set data, parse a set of test tasks, send the test tasks to the device management agent, and then switch to the waiting state; In the waiting state, if the test log returned by the device management agent is received, it will be converted to processing state B: In processing state B, a dedicated prompt word is designed to make the test execution agent perform the following steps: parse the test log, call the image test result judgment tool and the log test result judgment tool to determine whether the test has passed, call the test report generation tool to generate a test report for this round of testing, return the test report to the user, and finally transition to the completion state; In the completed state, when the user confirms the end of this round of testing, the state transitions from the completed state to the idle state.

8. The AI agent-based automated testing method for driver software according to claim 7, wherein: The device management agent has three states, namely processing state C, completion state and idle state; The device management agent is initially in the idle state. After receiving the test task sent by the test execution agent, it switches to the processing state C. In processing state C, a dedicated prompt word is designed to make the device management agent perform the following steps: parse the test target machine information corresponding to each test task, obtain the status of the test target machine, and arrange the test task execution order; When each test task is executed, the artifact pulling tool is called to pull the test artifacts; Send test artifacts and test instructions to the test target machine; collect test logs and test result data, and return them to the test execution agent, converting them to the completed state; In the completed state, when the user confirms the end of this round of testing, the state transitions from the completed state to the idle state.

9. The AI agent-based automated testing method for driver software according to claim 8, wherein: The two AI agents communicate using HTTP protocol, and each AI agent maintains a message queue; The process of the test execution agent sending information to the device management agent is as follows: the test execution agent adds the test task data to the message queue and attempts to send it immediately; if a success message is received, the data is removed from the message queue; if a failure message is received, it is resent after a preset time. If it still fails after the maximum number of retries, the test task is discarded and a test failure result is output; The process of the device management agent receiving and processing information sent by the test execution agent is as follows: first, a status check is performed to determine whether the current test execution agent is in a waiting state and whether the device management agent is in an idle state. If both are true, it indicates that the status is correct, and the test task data is added to the message queue of the device management agent. The test task is processed according to the process and a successful processing response is returned; otherwise, it indicates that the status is wrong and an error response is returned.

10. An AI-based automated testing platform for driver software, characterized in that: The automated testing platform includes a test execution agent, a device management agent, a hardware resource pool composed of target testers, a test database, a product library, and an AI tool set, wherein the test execution agent and the device management agent are implemented in the form of AI agents; The test execution agent is used to receive and parse the test input files sent by the user, then obtain the test case set data from the test database and generate a set of test tasks. It is also used to call the AI tool set to determine whether the test passes after receiving the test result data of each test task, and to call the AI tool set to generate a test report after all tests are completed. It is also used to upload the test log, test report and all test result data to the artifact library and return the test report to the user. The device management agent is used to arrange the test task execution order according to the status of the target test machine corresponding to each test task, call the AI tool set according to the current test task requirements, pull test artifacts from the artifact library, and finally send the test artifacts and test instructions to the test target machine; and for collecting test logs and test result data and sending them to the test execution agent; The test target machine is used to execute test actions according to test instructions and finally return test logs and test result data.

Images