AI skill natural language test scheduling method

Abstract

The application provides an AI skill natural language test scheduling method, which comprises the following steps: loading a preset AI skill by using an AI programming tool; automatically analyzing and converting a natural language test request, a defect verification request or a regression test request input by a tester or a developer into a standardized structured task parameter; then detecting a connection state and an available state of a local Android device, matching and distributing test resources to complete task execution; sequentially scheduling device resources to run an automated test framework, recording task states, logs, report paths and test results throughout the process; and finally returning the results through a task number polling mechanism, converting the structured test data into a natural language summary and feeding back the natural language summary to the AI programming tool. The application has the beneficial effect of forming a full-process scheduling system of a natural language input-parameter conversion-automatic routing-task execution-result closed loop, which greatly reduces the test and development self-test threshold.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing technology, and in particular to a method for scheduling AI skill natural language tests. Background Technology

[0002] AI skills refer to skill rules, prompts, script descriptions, or tool call instructions that can be loaded by AI programming tools. These skills are used to convert users' natural language testing intentions into structured parameters and commands that can be executed by the automated testing framework. AI programming tools refer to intelligent coding or intelligent agent tools that can understand natural language and perform engineering tasks according to context. These include, but are not limited to, Kiro, Claude Code, and OpenCode, and can also be other AI tools with natural language understanding capabilities. Automatic routing refers to the process of automatically selecting a local or remote execution path based on the local device's connection status, test project type, remote service availability, and device idle status.

[0003] With the rapid development of the mobile internet industry, the iteration speed of Android applications continues to accelerate. Automated testing has become a core means to ensure application quality and improve R&D efficiency, and Android application automated testing platforms have been widely adopted by various R&D teams. Currently, most mainstream Android application automated testing is built on testing frameworks such as pytest. In the conventional testing process, testers and developers need to manually open command-line tools, manually find the storage path of test scripts, memorize complex pytest tags and various execution parameters, manually configure the Android device number under test, and select the corresponding execution environment according to the testing requirements. The entire operation process is cumbersome and has scattered entry points, which is extremely unfriendly to newly hired testers and developers lacking testing experience. In practice, problems such as missing parameters, incorrect command input, and incorrect device selection are very likely to occur, directly leading to test task failure and additional rework costs.

[0004] For developers, in their daily R&D work, fixing program defects, locating online issues, and before and after submitting code versions all require quick verification to confirm whether functional problems have been reproduced, whether code modifications have introduced regression defects, and whether automated test cases for modules have passed correctly. Traditional automated testing models require developers to be proficient in the test framework directory structure, test case selection rules, command-line parameter meanings, device selection logic, and test report viewing methods. This significantly raises the technical threshold for self-testing, lengthens the closed-loop cycle of problem location and defect fixing, and severely impacts overall R&D efficiency.

[0005] In recent years, AI programming tools with natural language understanding capabilities have been gradually integrated into the entire R&D and testing process. Staff generally hope to directly describe testing requirements, defect verification requirements, and regression testing requirements using natural language, and use AI tools to automatically generate test commands and execute tasks. However, existing ordinary AI programming tools can only output command suggestions in text form and have not established a stable communication link with automated testing frameworks, physical testing equipment, and remote execution services. Therefore, they cannot directly drive the execution of test tasks, and the auxiliary value of AI tools cannot be fully realized.

[0006] Meanwhile, existing automated testing platforms suffer from numerous systemic flaws: when staff are not connected to the Android device under test on their own machines, they must manually switch to a remote machine or manually log in to the CI platform to initiate test tasks; staff need to manually check the running status of remote services and whether remote devices are idle, resulting in numerous manual intervention steps and extremely low automation; a unified parameter model is not established between natural language requests, automated test parameters, local execution logic, and remote execution logic, leading to chaotic data flow; during test task execution, status queries, log synchronization, result feedback, and exception recovery lack a unified link, and AI tools cannot achieve a complete closed loop from user requirement input to test result output. These issues hinder the development of Android automated testing towards intelligence, lightweighting, and efficiency, and the industry urgently needs a testing method that can combine AI capabilities to achieve fully automated scheduling throughout the entire process. Summary of the Invention

[0007] To address the problems in existing technologies, this invention provides an AI skill natural language testing scheduling method. This method enables the formation of a closed-loop scheduling system encompassing natural language input, parameter conversion, automatic routing, task execution, and results. It significantly lowers the barriers to testing and development self-testing, improves the efficiency and stability of automated testing, and can be widely applied to scenarios such as Android application automated testing platforms and AI-assisted development testing and verification platforms. This solves the problems of cumbersome operations, difficulty in memorizing command parameters, the need for manual switching between devices and execution environments, poor integration between AI tools and testing frameworks, and non-closed-loop task chains in traditional Android automated testing.

[0008] This invention provides an AI skill natural language testing scheduling method, applied to Android automated testing scenarios. It utilizes AI programming tools, a local testing framework, and a remote execution service to schedule test tasks, and includes the following steps: Step 1: Deploy AI skill files in the skill catalog of the AI ​​programming tool in advance. After the AI ​​programming tool starts or reloads, it reads the AI ​​skill file and loads the corresponding skill rules, parameter definitions and calling logic. Step 2: Receive natural language test requests or question verification requests input by the user in the AI ​​programming tool; Step 3: Based on the loaded AI skill rules, analyze the user's testing intent, convert the natural language test request or question verification request into standardized structured task parameters. If any parameters are missing, read the project's default configuration and automatically complete them. Step 4: Detect the connection status of the local Android device through the automatic routing module, filter and determine whether there is a local Android device in a valid and executable state; Step 5: When a native Android device in a valid executable state exists, call the local automated testing framework to execute the test task and obtain a unique task number; Step 6: After the test task is completed, release the occupied Android device resources and save the complete test results; Step 7: The AI ​​programming tool or local client polls the task status according to the task number. After the task is completed, it obtains the complete test results, generates a natural language readable summary, and feeds it back to the user, thus completing the test task loop.

[0009] The present invention is further improved by including the following step in step 4 when no native Android device in a valid executable state exists: Step 401: Read the preset remote service configuration list, check the health status of the remote execution service and the idle status of the Android devices under the remote execution service in turn, filter out the Android devices with normal status and available idle devices, submit the structured task parameters to the remote execution service, and obtain a unique task number. Step 402: After the remote execution service receives the task, it writes the task into the task queue and stores it persistently. The background thread retrieves the tasks in the queue order and matches them with the corresponding type of Android device resources. Step 403: The remote execution service constructs an automated test execution command based on the structured task parameters, calls the automated test framework to run test cases, records the task status, running logs, exit codes, report paths and test summary data in real time during execution, and then executes step 6.

[0010] In a further improvement to this invention, in step 1, the AI ​​skill file is in the form of a folder, which contains a skill description file, a parameter definition document, a calling example, and usage precautions; the AI ​​programming tool is an intelligent coding tool with natural language understanding and local tool calling capabilities, including any one or more of Kiro, Claude Code, and OpenCode.

[0011] In a further improvement to this invention, in step 3, the standardized structured task parameters include at least six fields: test case selection criteria, test path, execution round, project type, report type, and requester identifier.

[0012] In a further improvement to this invention, in step 4, the detection method for the local Android device is that the automatic routing module calls the Android debug bridge device list command, and only the Android devices that are connected normally and not occupied are determined as the local Android devices in a valid and executable state.

[0013] In a further improvement to the present invention, in step 401, the remote execution service can be deployed on the user's local machine or an independent remote computer, and the selection strategy of the remote execution service includes any one of the following: priority matching by project type, round-robin scheduling, load balancing matching, and device capacity matching.

[0014] In a further improvement to this invention, in step 402, the task persistent storage method adopts any one of JSON file, SQLite database, Redis, or relational database; when the remote execution service cannot obtain matching Android device resources, the task is marked as waiting or execution failed, and the task status is synchronously updated to persistent storage.

[0015] In a further improvement to this invention, in step 403, the execution entry point of the automated testing framework is any one of the following: the pytest entry script, an independent test script, or a CI pipeline task.

[0016] In a further improvement to this invention, in step 6, the feedback format of the test results includes any one or more of the following: terminal text, structured JSON data, test report link, email summary, and natural language summary.

[0017] In a further improvement to this invention, in step 4, the communication method between the remote execution service and the AI ​​programming tool and the local client adopts any one or more of the following: HTTP interface, gRPC, message queue, WebSocket, and CI platform interface.

[0018] The beneficial effects of this invention are as follows: This invention provides an AI skill natural language testing scheduling method. By loading preset AI skills into an AI programming tool, it automatically parses and converts natural language test requests, defect verification requests, or regression test requests input by testers and developers into standardized structured task parameters. Subsequently, an automatic routing module detects the connection and availability status of the local Android device, prioritizing the local device for test task execution. When no local device is available, it automatically retrieves the health status and device idle status of the remote execution service, matches and allocates remote test resources to complete the task execution. After receiving the task, the remote execution service performs queue management, persistent storage, and orderly scheduling of device resources to run the automated testing framework throughout the entire process. It records task status, logs, report paths, and test results; finally, it completes the result feedback through a task number polling mechanism, converting structured test data into natural language summaries and feeding them back to the AI ​​programming tool. This forms a closed-loop scheduling system of natural language input, parameter conversion, automatic routing, task execution, and result, which significantly reduces the threshold for testing and development self-testing, improves the efficiency and stability of automated test execution, and can be widely used in scenarios such as Android application automated testing platforms and AI-assisted development testing and verification platforms. It solves the problems of cumbersome operation, difficulty in remembering command parameters, need for manual switching of devices and execution environments, poor linkage between AI tools and testing frameworks, and non-closed-loop task links in existing technologies for traditional Android automated testing. Attached Figure Description

[0019] Figure 1 This is a flowchart of an AI skill natural language testing scheduling method according to the present invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0021] Please see Figure 1 This invention discloses an AI skill natural language testing scheduling method, applied to Android automated testing scenarios. It utilizes AI programming tools, a local testing framework, and a remote execution service to schedule test tasks, and includes the following steps: Step 1: Deploy AI skill files in the skill catalog of the AI ​​programming tool beforehand. After the AI ​​programming tool starts or reloads, it reads the AI ​​skill file and loads the corresponding skill rules, parameter definitions, and calling logic. The AI ​​skill file is in the form of a folder, which contains a skill description file, parameter definition document, calling examples, and usage precautions. The AI ​​programming tool is an intelligent coding tool with natural language understanding and local tool calling capabilities, including any one or more of Kiro, ClaudeCode, and OpenCode.

[0022] Step 2: Receive natural language test requests or question verification requests entered by the user in the AI ​​programming tool.

[0023] Step 3: Based on the loaded AI skill rules, analyze the user's testing intent and convert the natural language test request or issue verification request into standardized structured task parameters. If any parameters are missing, read the project's default configuration and automatically complete them. The standardized structured task parameters include at least six fields: test case selection criteria, test path, execution round, project type, report type, and requester identifier.

[0024] Step 4: The automatic routing module detects the connection status of the local Android devices, filters and determines whether there are any local Android devices in a valid executable state. The detection method for local Android devices is that the automatic routing module calls the Android debug bridge device list command, and only Android devices that are connected normally and not occupied are determined as local Android devices in a valid executable state.

[0025] Step 5: When a valid executable local Android device exists, the local automated testing framework is invoked to execute the test task and a unique task number is obtained. When no valid executable local Android device exists, the following steps are also included: Step 401: Read the preset remote service configuration list, sequentially verify the health status of the remote execution service and the idle status of the Android devices under the remote execution service, filter out Android devices with normal status and available idle devices, submit the structured task parameters to the remote execution service, and obtain a unique task number. The remote execution service can be deployed on the user's local machine or an independent remote computer. The selection strategy for the remote execution service includes any one of the following: priority matching by project type, round-robin scheduling, load balancing matching, and device capability matching. Step 402: After receiving the task, the remote execution service will... Tasks are written to a task queue and persistently stored. A background thread retrieves tasks in queue order and matches them with corresponding Android device resources. The task persistence storage method can be any one of JSON files, SQLite databases, Redis, or relational databases. When the remote execution service cannot obtain a matching Android device resource, the task is marked as waiting or has failed, and the task status is synchronously updated to persistent storage. In step 403, the remote execution service constructs automated test execution commands based on structured task parameters, calls the automated test framework to run test cases, and records task status, execution logs, exit codes, report paths, and test summary data in real time during execution, and then executes step 6. The execution entry point of the automated test framework can be any one of the following: the pytest entry script, an independent test script, or a CI pipeline task. Communication between the remote execution service and the AI ​​programming tool and local client uses any one or more of the following methods: HTTP interface, gRPC, message queue, WebSocket, and CI platform interface.

[0026] Step 6: After the test task is completed, release the occupied Android device resources and save the complete test results. The test results can be fed back in one or more of the following formats: terminal text, structured JSON data, test report link, email summary, and natural language summary.

[0027] Step 7: The AI ​​programming tool or local client polls the task status according to the task number. After the task is completed, it obtains the complete test results, generates a natural language readable summary, and feeds it back to the user, thus completing the test task loop.

[0028] In this embodiment, the present invention also provides an AI skill natural language testing scheduling system, including an AI skill definition module, a natural language intent parsing module, a standardized task parameter module, a local device detection module, an automatic routing module, a local execution module, a remote execution service module, a task status polling module, and a result feedback module.

[0029] During system runtime, testers or developers input natural language test or verification requests into the AI ​​programming tool. After loading pre-placed AI skill files in its skill directory, the AI ​​programming tool identifies the test or verification intent based on skill rules and converts the natural language request into structured task parameters including fields such as test case tags, test path, execution rounds, project type, report type, requester, and target service. Subsequently, the automatic routing module detects the connection status of the local Android device. If an available Android device exists, it directly calls the local automated test entry point for execution; if no available Android device exists, it queries the availability and idle status of the remote execution service based on the project type and remote service configuration, and selects a suitable remote execution service to submit the task.

[0030] After receiving a task, the remote execution service generates a task number, writes the task to a queue and persistent storage, acquires available device resources sequentially, constructs automated test execution commands, and calls the test framework to execute test cases. During execution, the service records the task status, exit code, log tail, report path, and test summary results. The AI ​​programming tool or local client polls the status according to the task number and obtains structured results after the task is completed, then feeds them back to testers or developers in natural language or structured summary form.

[0031] The AI ​​skills are loaded and triggered by natural language. AI skills are placed in folders within the skill directory of the AI ​​programming tool. These folders contain skill descriptions, parameter definitions, usage examples, and precautions. After the AI ​​programming tool is reloaded, testers or developers can directly input natural language requests, such as executing a specific type of smoke test case, specifying a test module, specifying execution rounds, specifying a project type, or requesting rapid verification of automated test cases corresponding to a fixed issue. The AI ​​programming tool is not limited to a specific product; Kiro, Claude Code, and OpenCode are only optional examples.

[0032] Structured parameter generation: AI skills generate unified task parameters based on test requests, including at least test case selection criteria or test paths, execution rounds, project type, report type, and requester identifier. For unprovided parameters, the system can read configuration files or project default values ​​to complete them. Structured parameters ensure that natural language requests do not directly depend on a fixed command, but are first converted into stable data objects, and then the final command is constructed by the local execution module or remote execution service.

[0033] The automatic routing module first detects the local Android device. During detection, it can call the Android debug bridge device list command and only consider devices in the executable connection state as valid devices. When a valid device exists on the local machine, the system prioritizes local execution; when no valid device exists on the local machine, the automatic routing module reads the remote execution service list from the configuration file, checks the service health status, queries whether there are idle devices on the remote machine, and selects a remote execution service that meets the conditions to submit the task.

[0034] The remote execution service can be deployed on the tester's or developer's local machine or on a remote computer. Before use, the server program needs to be started, and the Android device to be tested needs to be plugged into or connected to that computer. After receiving the task, the server generates a unique task number and records the request parameters, requester, creation time, and task status.

[0035] For task execution and result feedback, the server sets up a task queue. Once a task enters the queue, background worker threads retrieve tasks sequentially and attempt to acquire device resources that meet the project type requirements. Upon successful acquisition, the server constructs an automated test execution command and calls the test framework's entry script to execute the test. After execution, the server saves the execution status, exit code, log end, report path, and test result statistics. Clients or AI programming tools can query the task status by task number. Upon task completion, the server returns task conclusions, report path, log summary, and test statistics.

[0036] As can be seen from the above, compared with the prior art, the present invention has at least the following beneficial effects: 1. Testers or developers do not need to remember complex commands and parameters; they can directly trigger automated testing tasks through natural language.

[0037] 2. Developers can quickly and automatically verify defect fixes, feature changes, and regression risks within the AI ​​programming tool without needing to additionally understand the test case execution methods, parameter meanings, and execution entry points within the test framework.

[0038] 3. AI skills convert natural language intent into structured task parameters, reducing the execution risk caused by the instability of AI-generated commands.

[0039] 4. The system can automatically determine the status of the local Android device and automatically select a remote execution service when there is no local Android device, reducing the cost of manually switching environments.

[0040] 5. The remote execution service can be deployed on the local machine or a remote computer, adapting to personal testing, development self-testing, team public testing machines, and centralized equipment environments.

[0041] 6. A closed loop is formed from natural language requests to test result feedback through task numbering, status polling, and result feedback mechanisms.

[0042] 7. Through configuration management, it supports multiple project types, multiple remote services, and multiple execution parameters, improving the reusability of the testing framework in different projects.

[0043] 8. This solution focuses on improving the technical connection between AI tools and automated test execution systems, rather than simply displaying test reports or manually compiling test results.

[0044] The specific embodiments described above are preferred embodiments of the present invention and are not intended to limit the specific scope of the present invention. The scope of the present invention includes, but is not limited to, these specific embodiments. All equivalent changes made in accordance with the present invention are within the protection scope of the present invention.

Claims

1. A method for scheduling AI skill natural language tests, characterized in that, This approach, applied to Android automated testing scenarios, leverages AI programming tools, a local testing framework, and remote execution services to schedule test tasks, including the following steps: Step 1: Deploy AI skill files in the skill catalog of the AI ​​programming tool in advance. After the AI ​​programming tool starts or reloads, it reads the AI ​​skill file and loads the corresponding skill rules, parameter definitions and calling logic. Step 2: Receive natural language test requests or question verification requests input by the user in the AI ​​programming tool; Step 3: Based on the loaded AI skill rules, analyze the user's testing intent, convert the natural language test request or question verification request into standardized structured task parameters. If any parameters are missing, read the project's default configuration and automatically complete them. Step 4: Detect the connection status of the local Android device through the automatic routing module, filter and determine whether there is a local Android device in a valid and executable state; Step 5: When a native Android device in a valid executable state exists, call the local automated testing framework to execute the test task and obtain a unique task number; Step 6: After the test task is completed, release the occupied Android device resources and save the complete test results; Step 7: The AI ​​programming tool or local client polls the task status according to the task number. After the task is completed, it obtains the complete test results, generates a natural language readable summary, and feeds it back to the user, thus completing the test task loop.

2. The AI ​​skill natural language test scheduling method as described in claim 1, characterized in that: In step 4, if no native Android device in a valid executable state exists, the following steps are also included: Step 401: Read the preset remote service configuration list, check the health status of the remote execution service and the idle status of the Android devices under the remote execution service in turn, filter out the Android devices with normal status and available idle devices, submit the structured task parameters to the remote execution service, and obtain a unique task number. Step 402: After the remote execution service receives the task, it writes the task into the task queue and stores it persistently. The background thread retrieves the tasks in the queue order and matches them with the corresponding type of Android device resources. Step 403: The remote execution service constructs an automated test execution command based on the structured task parameters, calls the automated test framework to run test cases, records the task status, running logs, exit codes, report paths and test summary data in real time during execution, and then executes step 6.

3. The AI ​​skill natural language test scheduling method as described in claim 2, characterized in that: In step 1, the AI ​​skill file is in the form of a folder, which contains a skill description file, parameter definition document, calling examples, and usage precautions; the AI ​​programming tool is an intelligent coding tool with natural language understanding and local tool calling capabilities, including any one or more of Kiro, Claude Code, and OpenCode.

4. The AI ​​skill natural language test scheduling method as described in claim 3, characterized in that: In step 3, the standardized structured task parameters include at least six fields: test case selection criteria, test path, execution round, project type, report type, and requester identifier.

5. The AI ​​skill natural language test scheduling method as described in claim 4, characterized in that: In step 4, the detection method for the local Android device is that the automatic routing module calls the Android debug bridge device list command, and only Android devices that are connected normally and not occupied are determined as valid and executable local Android devices.

6. The AI ​​skill natural language test scheduling method as described in claim 5, characterized in that: In step 401, the remote execution service can be deployed on the user's local machine or an independent remote computer, and the selection strategy for the remote execution service includes any one of the following: priority matching by project type, round-robin scheduling, load balancing matching, and device capacity matching.

7. The AI ​​skill natural language test scheduling method as described in claim 6, characterized in that: In step 402, the task persistent storage method adopts any one of JSON file, SQLite database, Redis, or relational database; when the remote execution service cannot obtain matching Android device resources, the task is marked as waiting or execution failed, and the task status is synchronously updated to persistent storage.

8. The AI ​​skill natural language test scheduling method as described in claim 7, characterized in that: In step 403, the execution entry point of the automated testing framework is any one of the following: the pytest entry script, an independent test script, or a CI pipeline task.

9. The AI ​​skill natural language test scheduling method as described in claim 8, characterized in that: In step 6, the feedback format of the test results includes any one or more of the following: terminal text, structured JSON data, test report link, email summary, and natural language summary.

10. The AI ​​skill natural language test scheduling method as described in claim 9, characterized in that: In step 4, the communication between the remote execution service and the AI ​​programming tool and the local client adopts any one or more of the following methods: HTTP interface, gRPC, message queue, WebSocket, and CI platform interface.

Images