Mobile application GUI automatic testing method and system based on hierarchical reinforcement learning

Abstract

The invention provides a mobile application GUI automatic testing method and system based on hierarchical reinforcement learning, and belongs to the technical field of mobile application testing. A real hardware device or a virtual machine is used for building a testing environment and configuring an automatic exploration framework, a to-be-tested application program is initialized, GUI layer information and control layer information are extracted respectively, and the GUI layer information and the control layer information are extracted respectively; the method comprises the following steps: acquiring a GUI layer and a control layer, converting the GUI layer and the control layer into a fine-grained state vector convenient for subsequent reinforcement learning processing, exploring the GUI layer and the control layer by using hierarchical reinforcement learning in combination with a reward function, processing interaction between the two layers of networks, and performing conversion by using an FSM when an exception occurs. Respectively obtaining a code coverage rate and a crash detection result by utilizing the code coverage rate detection framework and the log information of Appium; the problem that an existing mobile application GUI automatic testing method does not consider rough GUI information granularity and too single exploration level, and consequently exploration path coverage is not comprehensive is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile application testing, and in particular, relates to a mobile application GUI automated testing method and system based on hierarchical reinforcement learning. Background Art

[0002] With the widespread adoption of mobile devices, smartphones and mobile applications have permeated every aspect of our lives. Mobile operating systems, due to their lightweight and flexible nature, have become a mainstream platform. As the primary medium for user interaction with applications, testing the graphical user interface (GUI) of mobile applications is directly related to software stability, reliability, and security. To discover potential defects, improve code coverage, and ensure user experience, efficient and comprehensive GUI testing has become a hot topic of research.

[0003] In recent years, with the rapid development of machine learning technology, mobile application GUI testing methods based on machine learning, especially reinforcement learning, have gradually become a research hotspot, demonstrating powerful adaptive exploration capabilities in the field of automated testing. Some methods, such as SwiftHand and MUBot, collect and analyze large amounts of real-world user interaction data to model and predict GUI behavior, thereby generating test sequences that closely resemble human operations. Although these methods rely on difficult-to-obtain labeled data, they have improved the intelligence level of testing to a certain extent. Furthermore, reinforcement learning, as a key branch of machine learning, has rapidly become a major research direction in GUI testing due to its ability to autonomously learn through interaction with the environment without requiring pre-labeled data. DroidbotX uses a Q-table to record the reward relationship between states and actions, guiding test behavior by continuously updating Q-values. However, in large-scale state spaces, Q-tables often suffer from the curse of dimensionality. To this end, methods based on deep reinforcement learning have emerged. DeepGUIT, for example, introduces a deep neural network to approximate the Q function and uses an experience replay mechanism to improve learning, enabling the test agent to more accurately evaluate the optimal action in each state. Wuji combines deep reinforcement learning with evolutionary algorithms, demonstrating its applicability in complex real-time environments using the example of automated testing of online fighting games. Furthermore, ARES employs a variety of advanced network structures, including Deep Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG), Double-Delayed DDPG (TD3), and Soft Actor-Critic (SAC), to handle continuous and high-dimensional action space problems, enabling more efficient exploration of deeper GUI states during testing. The OAT method utilizes Monte Carlo Tree Search (MCTS) to guide the test agent's global state, improving exploration efficiency.

[0004] Although these machine learning-based testing methods have made significant progress in exploring unknown states and automatically generating test sequences, they still have problems such as failing to fully utilize GUI fine-grained information, having too single an exploration level, and being prone to falling into repeated exploration and local optimal states. Summary of the Invention

[0005] To address the shortcomings of existing machine learning testing methods, namely, coarse granularity of GUI information, overly simple exploration levels, and a tendency to fall into repeated exploration and local optimal states, this paper proposes a mobile application GUI automated testing method and system based on hierarchical reinforcement learning through task decomposition, fine-grained state modeling, hierarchical reinforcement learning, a dynamic reward mechanism, and a finite state machine (FSM).

[0006] The present invention is implemented through the following technical solution: a mobile application GUI automated testing method based on hierarchical reinforcement learning: the method specifically comprises the following steps:

[0007] Step 1: Use real hardware devices or virtual machines to build a test environment and configure the automated testing framework to initialize the application under test;

[0008] Step 2: Use the framework built in step 1 to extract the GUI layer and control layer information respectively, and convert them into fine-grained state vectors that are convenient for subsequent reinforcement learning processing;

[0009] Step 3: Based on the vector extracted in step 2, hierarchical reinforcement learning is used in combination with the reward function to explore the GUI layer and the control layer respectively, and the interaction between the two layers of the network is processed. When an exception is encountered, the FSM is used for conversion;

[0010] Step 4: Use the code coverage detection framework and Appium log information to obtain code coverage and crash detection results respectively.

[0011] Furthermore, in step 1,

[0012] Step 1.1: Use a virtual machine or real hardware device to build a test platform and install the corresponding system;

[0013] Step 1.2: Install the Appium testing framework and configure the Android Debug Bridge.

[0014] Step 1.3: Install and initialize the mobile application.

[0015] Furthermore, in step 2,

[0016] Step 2.1: Use Appium and ADB in step 1 to extract the GUI information in the XML of the mobile application, process it in a fine-grained manner, and convert it into a corresponding vector representation;

[0017] Step 2.2: Use Appium and ADB in step 1 to extract the control information in the XML of the mobile application, process it in a fine-grained manner, and convert it into the corresponding vector representation;

[0018] Step 2.3: Integrate the vector representations extracted in steps 2.1 and 2.2 and record the visits to each state to facilitate dynamic calculation of rewards.

[0019] Furthermore, in step 3,

[0020] Step 3.1: Input the GUI vector representation extracted in step 2.1 into the high-level Actor-Critic network. Select the GUI interface to be explored based on the dynamic reward function, convert it into an executable action and hand it over to Appium for operation and execution. The status feedback after the operation is collected in real time.

[0021] Step 3.2: Input the control vector representation extracted in step 2.2 into the low-level Actor-Critic network. Select the control to be explored based on the dynamic reward function, convert it into an executable action, and hand it over to Appium for operation and execution. The status feedback after the operation is collected in real time.

[0022] Step 3.3: Use FSM to store operation traces to record each test state and its transition path, so as to facilitate timely transition when continuous local optimality or long sequence operation failure is detected;

[0023] Step 3.4: Synchronously update high-level and low-level network parameters and interact with each other to continuously optimize the decision-making strategy.

[0024] Furthermore, in step 3.1, a dynamic reward function is designed for the GUI hierarchy based on the extracted fine-grained state vector: GUIs with more controls are preferentially visited;

[0025] In step 3.2, a dynamic reward function is designed for the control hierarchy based on the extracted fine-grained state vector: controls that have not been explored or have been visited less frequently are given priority;

[0026] More crash detections are found by sharing the same reward function between the GUI layer and the control layer.

[0027] Furthermore, in step 3.3, the FSM module is first constructed. Each node in the FSM is used to record each test state, and its relationship represents the transition path between two states;

[0028] When a continuous local optimal state is detected, the shortest path out of the local optimal state is calculated through the graph constructed by the FSM, and the shortest path back to the less frequently visited state is returned to continue the exploration;

[0029] When a long sequence operation failure is detected, the shortest distance from the exploration failure state to the node avoiding this state is calculated based on the graph constructed by FSM, and state transition is performed to avoid re-exploring from the initial node.

[0030] Furthermore, in step 4,

[0031] Step 4.1: Use the code coverage detection tool to obtain the code coverage results based on the exploration path in step 3, and analyze the log information in Appium to count the number of application crash detections;

[0032] Step 4.2: Get mobile app test coverage and crash detection count.

[0033] A testing system for mobile application GUI automated testing method based on hierarchical reinforcement learning,

[0034] The test system includes: a configuration module, a state information extraction module, a hierarchical reinforcement learning module and a test acquisition module;

[0035] The configuration module uses real hardware devices or virtual machines to build a test environment and configure an automated testing framework to initialize the application to be tested;

[0036] The state information extraction module uses the framework built by the configuration module to extract the GUI layer and control layer information respectively, and converts them into fine-grained state vectors that are convenient for subsequent reinforcement learning processing;

[0037] The hierarchical reinforcement learning module uses hierarchical reinforcement learning to explore the GUI layer and the control layer respectively based on the vectors extracted by the state information extraction module and the reward function, and handles the interaction between the two layers of the network, and uses FSM to perform conversion when an exception is encountered;

[0038] The test acquisition module uses the code coverage detection framework and Appium's log information to obtain code coverage and crash detection results respectively.

[0039] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0040] A computer-readable storage medium is used to store computer instructions, which implement the steps of the above method when executed by a processor.

[0041] Beneficial effects of the present invention

[0042] Compared with the existing technology, the present invention proposes a mobile application GUI testing method based on hierarchical reinforcement learning. First, through fine-grained information extraction technology, the rich attributes, text information, interaction frequency and other data in the GUI information and controls are converted into state vectors; then, a hierarchical Actor-Critic network structure is adopted, in which the high-level network is responsible for the selection and exploration of the macro GUI interface and the large controls therein, and the low-level network performs fine operations on specific controls, realizing efficient decomposition and collaborative optimization of test tasks.

[0043] At the same time, this paper designs a dynamic reward function that, while fully considering state coverage and crash detection goals, combines extracted fine-grained information to provide reasonable incentives for the test agent, prompting it to continuously discover new states and conduct in-depth exploration during the testing process. This solves the problems that existing mobile application GUI automated testing methods fail to address, such as the coarse granularity of GUI information and the overly narrow exploration level, which leads to incomplete exploration path coverage. This can improve the code coverage rate and the number of crash detections in automated testing.

[0044] The present invention also fully considers the problem of falling into repeated exploration and local optimal state during the exploration process, uses a finite state machine to record the exploration state and convert it in time when necessary, thereby improving the exploration efficiency and test effect.

[0045] In addition, in order to avoid falling into local optimality and test interruption problems in long sequence operations, the present invention introduces a finite state machine to perform global path construction and path planning in the test process, so that when an abnormality occurs, it can quickly return to the optimal access state and re-plan the optimal test path.

[0046] In summary, the present invention significantly improves code coverage and crash detection capabilities, can increase code coverage and the number of crash detections in automated testing, and is suitable for mobile application GUI automated testing scenarios in the field of software engineering. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flowchart of the mobile application GUI testing method based on hierarchical reinforcement learning according to the present invention;

[0048] Figure 2 is a schematic diagram of the Markov process principle in reinforcement learning according to the present invention;

[0049] Figure 3 This is a model diagram of the mobile application GUI testing method based on hierarchical reinforcement learning described in the present invention. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0051] The experimental methods used in the following examples are conventional methods unless otherwise specified. The materials, reagents, methods, and instruments used are conventional in the art and can be obtained commercially by those skilled in the art unless otherwise specified.

[0052] like Figure 1 As shown, the present invention proposes a mobile application GUI automated testing method based on hierarchical reinforcement learning.

[0053] Step 1: Use real hardware devices or virtual machines to build a test environment and configure the automated testing framework to initialize the application under test (AUT);

[0054] Step 1.1: Use a real Android device or virtual machine to build a test platform, configure the operating system, and install the corresponding version of the Android system;

[0055] Step 1.2: Install the Appium test framework to operate the GUI and configure the Android Debug Bridge (ADB) to perform device control and command transmission;

[0056] Step 1.3: Install the Android app to be tested on the device or virtual machine, start the app, and ensure that its initial GUI state is displayed correctly.

[0057] Step 2: Use the framework built in step 1 to extract the GUI layer and control layer information respectively, and convert them into fine-grained state vectors that are convenient for subsequent reinforcement learning processing;

[0058] In order to fully explore each state in the GUI, information in the GUI level and the control level is extracted respectively, and the potential information in the high and low levels is fully obtained and converted into a vector representation that can be processed by the reinforcement learning algorithm.

[0059] Step 2.1: Use Appium and ADB in step 1 to extract the GUI information in the XML of the mobile application, process it in a fine-grained manner, and convert it into a corresponding vector representation;

[0060] For the GUI layer, each GUI state is represented as:

[0061] s gui =[id,(a0,...,am ),(w0,...,w n ),s gfre ,s wcount ]

[0062] Here, id is the unique identifier of each GUI interface, which is convenient for subsequent statistics of the access frequency of each GUI interface. (a0,...,a m ) is a one-hot encoding set that identifies whether the current activity is in progress. All activities occurring in the current GUI are represented as 1, otherwise they are marked as 0. (w0,...,w n ) contains all the control information in the current GUI interface. gfre Indicates the access frequency of the GUI, s wcount The number of controls contained in the current GUI.

[0063] Step 2.2: Use Appium and ADB in step 1 to extract the control information in the XML of the mobile application, process it in a fine-grained manner, and convert it into the corresponding vector representation;

[0064] For the control layer, the representation of each control state is:

[0065] s widget =[s id ,s loc ,s type ,s inter ,s wfre ]

[0066] Here, s id is a unique identifier for each control, s loc is the location information of the current control, s type Indicates the type information of the control. inter It is a 10-dimensional vector, each dimension indicates whether the control supports click, long press, input or scrolling operations. If the control supports the corresponding operation, the corresponding dimension is set to 1. wfre Indicates the access frequency of the current control.

[0067] Step 2.3: Integrate the vector representations extracted in steps 2.1 and 2.2 and record the visits to each state to calculate the dynamic reward;

[0068] Among them, in the fine-grained state vector extracted in step 2.1 and step 2.2, s gfre and widgetfre are used to record the access frequency of GUI and controls respectively.

[0069] Step 3: If Figure 2 、 3,According to the vector extracted in step 2, combined with the reward function, hierarchical reinforcement learning is used to explore the GUI layer and the control layer respectively, and the interaction between the two layers of the network is processed. When an exception is encountered, FSM is used for conversion;

[0070] In order to effectively avoid the exploration level being too single and easily falling into repeated exploration and local optimal state, the testing of GUI is handled by hierarchical reinforcement learning algorithm, supplemented by dynamic reward function and FSM to guide its exploration process.

[0071] Step 3.1: Input the GUI vector representation extracted in step 2.1 into the high-level Actor-Critic network. Select the GUI interface to be explored based on the dynamic reward function, convert it into an executable action and hand it over to Appium for operation and execution. The status feedback after the operation is collected in real time.

[0072] Among them, the present invention designs a dynamic reward function for the GUI layer based on the extracted fine-grained state vector:

[0073]

[0074] Where N1, R1, R2, R3 and R4 are constants, λ1 is a coefficient, and s wcount is the number of controls in the current GUI, s gfre is the access frequency of the current GUI. The present invention tends to give priority to accessing GUIs with more controls. Therefore, if a GUI has not been accessed before, it will be accessed based on the number of controls in the current GUI. wnumber Returns a larger reward. If the GUI has been visited, then the reward is calculated based on the current GUI's visit frequency s. gfre Returns a negative reward. If the current GUI does not belong to the AUT, then a negative reward is returned. Otherwise, a smaller negative reward is returned.

[0075] Step 3.2: Input the control vector representation extracted in step 2.2 into the low-level Actor-Critic network. Select the control to be explored based on the dynamic reward function, convert it into an executable action, and hand it over to Appium for operation and execution. The status feedback after the operation is collected in real time.

[0076] Among them, the present invention designs a dynamic reward function for the control level based on the extracted fine-grained state vector:

[0077]

[0078] Where R5, R6, R7 and R8 are constants, λ2 is a coefficient, and s wfreis the access frequency of the control. The present invention tends to give priority to accessing controls that have not been explored or have been accessed less frequently. Therefore, if a GUI has not been accessed before, a larger fixed value reward will be returned. If the current control has been accessed, the reward will be increased according to the access frequency of the current control s. wfre Returns a negative reward. If the current GUI does not belong to the AUT, then a negative reward is returned. Otherwise, a smaller negative reward is returned.

[0079] In addition, in order to find more crash detections, the GUI layer and the control layer share the same reward function:

[0080]

[0081] Where: crashTypes represents the number of crash types currently detected, N2, R9, R 10 and R 11 is a constant. This reward function favors discovering new crash types, so when an unseen crash type is detected, a larger reward is returned based on the number of crash types currently detected, crashTypes. If a previously detected crash type is encountered, a relatively smaller reward is returned. Otherwise, a small negative reward is returned.

[0082] Step 3.3: Use FSM to store operation traces to record each test state and its transition path, so as to facilitate timely transition when continuous local optimality or long sequence operation failure is detected;

[0083] First, an FSM module is constructed. Each node in the FSM is used to record each test state, and its relationship represents the transition path between two states. When a continuous local optimal state is detected, the shortest path out of the local optimal state is calculated through the graph constructed by the FSM, and the shortest path back to the low-frequency access state is returned to continue exploration. When a long sequence of operation failures is detected, the shortest distance from the exploration failure state to the node that avoids this state is calculated based on the graph constructed by the FSM, and a state transition is performed to avoid re-exploring from the initial node.

[0084] Step 3.4: Synchronously update high-level and low-level network parameters and interact with each other to continuously optimize the decision-making strategy.

[0085] The goal of the high-level network is to decide which GUI to explore or which more macro-level interface operation to perform, such as switching to another page, opening a menu, or opening a form. The low-level network selects more detailed control actions based on the actions of the upper-level network. Because a GUI typically contains multiple controls, a high-level exploration typically corresponds to multiple low-level explorations. The high-level network defines the exploration scope of the low-level network, and the results of the low-level network's operations (such as success, failure, or other feedback) are fed back to the high-level network as information, influencing the high-level network's future decisions. This feedback mechanism helps the system better understand which operations are effective and which may need to be reevaluated.

[0086] First, initialize the high-level Actor-Critic network H and the low-level Actor-Critic network L, which work together to effectively explore and test the GUI. At the same time, initialize the high-level playback buffer B high and the lower-level playback buffer B low , which is used to store the experience data of high- and low-level networks, thereby promoting efficient learning through the experience replay mechanism. In addition, the interaction weight parameter α is introduced to control the influence of low-level feedback on the high-level network.

[0087] In each iteration of the main loop, first get the current GUI state s high , and the high-level network H selects a GUI layer action a based on this state high After executing the selected action, the GUI will transition to the new state s' high .

[0088] Based on s high to s' high Transition and action a high , calculate the high-level reward r high The reward reflects the progress of the high-level task, such as discovering a new GUI state or a crash. At the same time, initialize the variable r feedback Zero is used to accumulate feedback from lower-level networks.

[0089] For high-level actions high For each associated control, the low-level network L performs detailed exploration. First, the current control state s is obtained low , and the lower layer network selects an action a based on the state low After executing the selected control action, the control will transition to the new state s' low . Calculate the low-level reward r low , to reflect the effectiveness of the control interaction. The reward and the corresponding state transition are stored in the low-level replay buffer B low It is used to train the low-level network L. The low-level rewards are aggregated through the defined feedback function and accumulated to rfeedback In this way, it affects the decision-making of high-level networks.

[0090] By feeding the lower layer feedback r feedback Multiply by the weight parameter α to update the high-level reward r high This mechanism ensures that the high-level network can comprehensively consider the results of low-level exploration and make more reasonable decisions. high ,a high ,r high ,s' high ) is stored in the high-level playback buffer B high It is used to train the high-level network H.

[0091] Finally, the high-level network H and the low-level network L are iteratively updated by sampling from their respective replay buffers to ensure continuous learning. The loop terminates when a predefined termination condition is met (e.g., code coverage and crash detection count no longer increase).

[0092] Step 4: Use the code coverage detection framework and Appium log information to obtain code coverage and crash detection results respectively.

[0093] Step 4.1: Use the code coverage detection tool to obtain the code coverage results based on the exploration path in step 3, and analyze the log information in Appium to count the number of application crash detections;

[0094] Step 4.2: Get mobile app test coverage and crash detection count.

[0095] A testing system for mobile application GUI automated testing method based on hierarchical reinforcement learning,

[0096] The test system includes: a configuration module, a state information extraction module, a hierarchical reinforcement learning module and a test acquisition module;

[0097] The configuration module uses real hardware devices or virtual machines to build a test environment and configure an automated testing framework to initialize the application to be tested;

[0098] The state information extraction module uses the framework built by the configuration module to extract the GUI layer and control layer information respectively, and converts them into fine-grained state vectors that are convenient for subsequent reinforcement learning processing;

[0099] The hierarchical reinforcement learning module uses hierarchical reinforcement learning to explore the GUI layer and the control layer respectively based on the vectors extracted by the state information extraction module and the reward function, and handles the interaction between the two layers of the network, and uses FSM to perform conversion when an exception is encountered;

[0100] The test acquisition module uses the code coverage detection framework and Appium's log information to obtain code coverage and crash detection results respectively.

[0101] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0102] A computer-readable storage medium is used to store computer instructions, which implement the steps of the above method when executed by a processor.

[0103] The memory in the embodiments of the present application can be volatile memory or non-volatile memory, or can include both volatile and non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to comprise, but not be limited to, these and any other suitable types of memory.

[0104] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired connection such as a coaxial cable, optical fiber, digital subscriber line (DSL), or wireless connection such as infrared, wireless, or microwave. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium such as a floppy disk, hard disk, magnetic tape, an optical medium such as a high-density digital video disc (DVD), or a semiconductor medium such as a solid-state disc (SSD).

[0105] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it will not be described in detail here.

[0106] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-described method embodiment can be completed by hardware integrated logic circuits in the processor or by software instructions. The above-described processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above-described method.

[0107] The above is a detailed introduction to the mobile application GUI automated testing method and system based on hierarchical reinforcement learning proposed in the present invention, and the principles and implementation methods of the present invention are explained. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A mobile application GUI automated testing method based on hierarchical reinforcement learning, characterized by: The method specifically comprises the following steps: Step 1: Use real hardware devices or virtual machines to build a test environment and configure the automated testing framework to initialize the application under test; Step 2: Use the framework built in step 1 to extract the GUI layer and control layer information respectively, and convert them into fine-grained state vectors that are convenient for subsequent reinforcement learning processing; Step 3: Based on the vector extracted in step 2, hierarchical reinforcement learning is used in combination with the reward function to explore the GUI layer and the control layer respectively, and the interaction between the two layers of the network is processed. When an exception is encountered, the FSM is used for conversion; Step 4: Use the code coverage detection framework and Appium log information to obtain code coverage and crash detection results respectively.

2. The method according to claim 1, wherein: In step 1, Step 1.1: Use a virtual machine or real hardware device to build a test platform and install the corresponding system; Step 1.2: Install the Appium testing framework and configure the Android Debug Bridge. Step 1.3: Install and initialize the mobile application.

3. The method according to claim 2, wherein: In step 2, Step 2.1: Use Appium and ADB in step 1 to extract the GUI information in the XML of the mobile application, process it in a fine-grained manner, and convert it into a corresponding vector representation; Step 2.2: Use Appium and ADB in step 1 to extract the control information in the XML of the mobile application, process it in a fine-grained manner, and convert it into the corresponding vector representation; Step 2.3: Integrate the vector representations extracted in steps 2.1 and 2.2 and record the visits to each state to facilitate dynamic calculation of rewards.

4. The method according to claim 3, wherein: In step 3, Step 3.1: Input the GUI vector representation extracted in step 2.1 into the high-level Actor-Critic network. Select the GUI interface to be explored based on the dynamic reward function, convert it into an executable action and hand it over to Appium for operation and execution. The status feedback after the operation is collected in real time. Step 3.2: Input the control vector representation extracted in step 2.2 into the low-level Actor-Critic network. Select the control to be explored based on the dynamic reward function, convert it into an executable action, and hand it over to Appium for operation and execution. The status feedback after the operation is collected in real time. Step 3.3: Use FSM to store operation traces to record each test state and its transition path, so as to facilitate timely transition when continuous local optimality or long sequence operation failure is detected; Step 3.4: Synchronously update high-level and low-level network parameters and interact with each other to continuously optimize the decision-making strategy.

5. The method according to claim 4, characterized in that: In step 3.1, a dynamic reward function is designed for the GUI hierarchy based on the extracted fine-grained state vector: GUIs with more controls are given priority; In step 3.2, a dynamic reward function is designed for the control hierarchy based on the extracted fine-grained state vector: controls that have not been explored or have been visited less frequently are given priority; More crash detections are found by sharing the same reward function between the GUI layer and the control layer.

6. The method according to claim 5, characterized in that: In step 3.3, first construct the FSM module. Each node in the FSM is used to record each test state, and its relationship represents the transition path between two states. When a continuous local optimal state is detected, the shortest path out of the local optimal state is calculated through the graph constructed by the FSM, and the shortest path back to the less frequently visited state is returned to continue the exploration; When a long sequence operation failure is detected, the shortest distance from the exploration failure state to the node avoiding this state is calculated based on the graph constructed by FSM, and state transition is performed to avoid re-exploring from the initial node.

7. The method according to claim 6, characterized in that: In step 4, Step 4.1: Use the code coverage detection tool to obtain the code coverage results based on the exploration path in step 3, and analyze the log information in Appium to count the number of application crash detections; Step 4.2: Get mobile app test coverage and crash detection count.

8. A testing system for executing the mobile application GUI automated testing method based on hierarchical reinforcement learning according to any one of claims 1 to 7, characterized in that: The test system includes: a configuration module, a state information extraction module, a hierarchical reinforcement learning module and a test acquisition module; The configuration module uses real hardware devices or virtual machines to build a test environment and configure an automated testing framework to initialize the application to be tested; The state information extraction module uses the framework built by the configuration module to extract the GUI layer and control layer information respectively, and converts them into fine-grained state vectors that are convenient for subsequent reinforcement learning processing; The hierarchical reinforcement learning module uses hierarchical reinforcement learning to explore the GUI layer and the control layer respectively based on the vectors extracted by the state information extraction module and the reward function, and handles the interaction between the two layers of the network, and uses FSM to perform conversion when an exception is encountered; The test acquisition module uses the code coverage detection framework and Appium's log information to obtain code coverage and crash detection results respectively.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images