Cross-platform reinforcement learning traversal testing method for mobile applications based on image understanding

Abstract

A cross-platform reinforcement learning traversal testing method for mobile applications based on image understanding includes an interaction module, an image understanding module, and a reinforcement learning module. The interaction module captures the running state of the APK and provides it to the image understanding module. The reinforcement learning module then selects actions to execute, thereby interacting with the mobile application. The image understanding module uses a screenshot encoder to capture and analyze the current application interface, generating feature vectors for encoding the state and executable actions. The reinforcement learning module uses a DQN model to analyze the pros and cons of state and action pairs, selects the optimal executable action, and achieves efficient exploration of the mobile application's state space.

Description

Technical Field

[0001] This invention belongs to the field of software testing. It performs automated traversal testing on provided mobile applications. At the front end, it extracts information by performing image understanding on screenshots. At the bottom layer, it uses deep reinforcement learning methods to update exploration strategies and generate test inputs, ultimately achieving the goal of detecting mobile application quality and discovering vulnerabilities. Background Art

[0002] With the rapid development of the internet and electronic devices in recent years, people's dependence on the internet has gradually increased. Mobile applications have become intertwined with every aspect of social life. Against the backdrop of the internet's rapid development, mobile applications across various platforms are rapidly being updated and iterated. However, under these circumstances, the quality of mobile applications is difficult to guarantee. Therefore, testing technology for mobile applications has become a hot topic in academia and industry.

[0003] Among mobile application testing methods, automatic traversal testing is a common type of automated testing. It aims to detect mobile application quality and discover vulnerabilities by covering as many mobile application scenarios as possible. Currently, mainstream automatic traversal testing includes random exploration strategies and model-based exploration strategies. Random exploration strategies generate random events in each state as input to the current state to achieve state transitions. Although widely used, they do not utilize the information gain obtained during the exploration process, often resulting in invalid events and repeated explorations, making it impossible to achieve efficient page coverage within a limited time. Model-based methods use prior knowledge to construct a state transition model corresponding to the software to generate test cases. However, they are highly dependent on the model, and the quality of the model largely determines the effectiveness of the test. At the same time, this method has strong limitations and lacks good generalization performance.

[0004] At the same time, existing test input generation techniques largely rely on platform characteristics: the platform provides interfaces for testers to obtain information. However, since these interfaces are directly tied to the platform and cannot be used universally across platforms, the cost of automated traversal testing increases. We aim to reduce this unnecessary cost and improve the platform interoperability of automated traversal testing techniques. Currently, most software interacts with users through a graphical user interface (GUI). A mainstream automated traversal testing technique is also based on this, using GUI content as a test basis. A technique that uses screenshots of mobile app pages to automatically generate crowdsourced test defect reports has also been proposed. Furthermore, among the latest technologies, reinforcement learning has been initially applied to automated traversal testing techniques using GUI interfaces. By continuously interacting with the environment and leveraging feedback from the environment on actions, it optimizes sequential action decisions and achieves extensive coverage of the state space. Reinforcement learning strategies can fully leverage their advantages in software testing. Appropriate feedback on behavior can continuously optimize the exploration decision process, overcoming the shortcomings of traditional automated testing exploration strategies in terms of insufficient coverage and achieving extensive and comprehensive coverage of the page. In the field of mobile application testing, some researchers have done some work, but the problem of platform universality has not yet been solved.

[0005] Based on the above work, this paper deeply explores the rich and intuitive information contained in mobile application interface images, fully leveraging the advantages of reinforcement learning in exploration tasks. Based on existing research results, corresponding image processing and understanding technologies and reinforcement learning methods are summarized, modified, and new technical methods are added. They are applied to control recognition and image understanding in mobile application interface images, as well as page exploration during testing, thereby automatically generating test inputs. Summary of the Invention

[0006] This invention addresses the problem that automated testing techniques for mobile applications cannot capture content embedded in HTML5 pages and canvas, resulting in poor overall testing results. Our invention addresses this issue by understanding the information contained in screenshots captured during mobile application runtime and adjusting the test exploration strategy accordingly.

[0007] The technical solution of the present invention is: a cross-platform reinforcement learning traversal testing method for mobile applications based on image understanding, which is characterized by being able to generate a test report based on the executable APK file provided by the user. The generation method includes the following three modules:

[0008] 1) Interaction Module: This module captures screenshots of the APK's running state for use by the image understanding module. It also executes actions selected by the reinforcement learning module within the APK. This module uses the adb command line for interaction.

[0009] 2) Image Understanding Module: This module understands the captured screenshots during testing, identifies and generates the control tree and corresponding action space within the page, and produces a state feature vector and an action feature vector, respectively, which serve as the state and action spaces for the reinforcement learning module. Feature extraction in this module uses OCR and Canny edge detection to identify text and graphical controls within the application interface and construct a control tree corresponding to the screenshot. Each node in the tree corresponds to a control, and the corresponding interactive actions are generated. After constructing the overall action space for the control tree, a CNN is used to extract its action feature vectors. An LSTM recurrent neural network is used to convert the control tree into a state feature vector for output.

[0010] 3) Reinforcement Learning Module: The state feature vector and action feature vector obtained by the image understanding module serve as the state and action spaces for reinforcement learning, respectively. These are then fed into the DQN neural network, which then outputs an interactive action selected by the neural network. The interaction module executes the final interactive action using an ε-greedy strategy. Newly captured screenshots, after feature extraction by the image understanding module, are used along with previous states and actions as inputs to the reward function to generate the corresponding reward. The state-action space and its reward values are used for experience replay and stored in the replay memory. The replay memory periodically selects a batch of samples for real-time updating of the DQN neural network.

[0011] The present invention is characterized in that:

[0012] 1. The first application of image understanding to capture the state of reinforcement learning strategies in mobile application testing

[0013] 2. First application of DQN to mobile application testing BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Flowchart for the implementation of the present invention.

[0015] Figure 2 The figure is the flow chart of the interactive module.

[0016] Figure 3 Flowchart of the image understanding module.

[0017] Figure 4 This is the flow chart of the reinforcement learning module. DETAILED DESCRIPTION

[0018] The embodiments of the present invention are described below. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification.

[0019] This patent implements cross-platform reinforcement learning traversal testing for mobile applications based on image understanding, primarily employing image understanding and deep reinforcement learning technologies. The key technologies involved include Canny, OCR, twin neural networks, recurrent neural networks (RNNs), and Deep Q-Network (DQN).

[0020] 1. Control Isolation Analysis

[0021] In this paper, we use Canny technology to segment app screenshots and extract each individual control in the image for analysis. The Canny edge detection algorithm first applies Gaussian filtering to smooth the image to remove noise; then finds the image's intensity gradient; then applies non-maximum suppression to eliminate false edge detections; then applies a double threshold method to determine possible boundaries; and finally, uses hysteresis to track boundaries.

[0022] 2. Recognize text information

[0023] In this paper, we use optical character recognition (OCR) technology to identify the rich textual information present in controls in application screenshots. OCR is the process by which an electronic device (such as a scanner or digital camera) examines printed characters, determines their shape by detecting dark and light patterns, and then uses character recognition methods to translate these shapes into computer text.

[0024] 3. Interface feature extraction

[0025] In this invention, we use a recurrent neural network (RNN) to isolate and analyze the control and the information obtained from text recognition, and further obtain the coordinate position information of the current page control through Group, Line, and Column operations to generate a control tree. The control tree is used as input and converted into a feature vector output. The recurrent neural network is a type of recursive neural network that takes sequence data as input, recursively in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain. The recurrent neural network has memory, parameter sharing, and Turing completeness, so it has certain advantages when learning the nonlinear characteristics of the sequence.

[0026] .4. Interface feature distinction

[0027] In this paper, we use a twin neural network architecture, using the edit distance between two control trees as the label. The Manhattan distance calculated from the LSTM output of the two control trees during interface feature extraction is used as the output, loss is calculated, and backpropagation is performed. The twin neural network consists of two neural networks with shared parameters. Two inputs are passed into each of the shared neural networks, mapped to a new space, and a representation of the input in the new space is formed. The similarity of the two inputs is evaluated by calculating the loss.

[0028] 5. Action feature extraction

[0029] In this invention, we use a convolutional neural network to extract features from executable actions and convert the control action into a feature vector. A convolutional neural network is a type of feedforward neural network with a deep structure that includes convolutional calculations. Convolutional neural networks have the ability to learn representations and can perform translation-invariant classification of input information based on their hierarchical structure. Convolutional neural networks are constructed to mimic the visual perception mechanism of organisms and can perform supervised and unsupervised learning. The shared convolution kernel parameters within the hidden layer and the sparsity of inter-layer connections enable convolutional neural networks to grid features with a small amount of computation.

[0030] 6. Reinforcement Learning Exploration

[0031] In this paper, we employ the Deep Q-Network approach. By extracting interface features, we obtain feature vectors that serve as inputs for states and actions, outputting corresponding predicted cumulative rewards with discount factors. Actions are then executed based on Boltzmann probability sampling. DQN uses a deep neural network to fit the Q function, updating it through methods such as a target network and experience replay to guide exploration strategies. By combining experience replay with reward function configuration, it addresses issues such as data correlation and non-static distribution. The introduction of a target network also improves stability.

[0032] 7. Reward function setting

[0033] In this invention, we adopt curiosity - Driven Rewards: where a t Indicates that the page can execute actions at time t, b t Indicates that the page has executed an action at time t, N t Indicates the state transition (s at time t t , a t , s t+1 ) occurs. When DQN is updated, the page exploration tendency is updated through this reward function.

[0034] The present invention's cross-platform reinforcement learning traversal testing method for mobile applications based on image understanding mainly includes:

[0035] Interaction module: Based on the executable APK file given by the user, this tool runs it, interacts with it, and generates an execution sequence.

[0036] Image understanding module: Use the screenshot encoder (including control isolation analysis, interface feature extraction, etc.) to take a screenshot of the current application interface and analyze it, generating feature vectors after encoding the state and executable actions.

[0037] Reinforcement learning exploration module: While recording the feature vectors and page transfer information encoded by the image understanding module, it updates the Deep Q-Network through real-time backpropagation to guide the exploration strategy and achieve large-scale and comprehensive coverage of the page.

Claims

1. A cross-platform reinforcement learning traversal testing method for mobile applications based on image understanding, characterized by Generate a test report based on the specified APK file provided by the user. The report contains the abnormal problems found in the test of the APK file. The working steps of the automated software traversal test system implemented based on this method are as follows: (1) The user uploads an APK file; (2) This method tests the files uploaded by the user, mainly including: (2.1) The interaction module uses debugging tools and a debugging platform to execute the interactive actions selected by the reinforcement learning module during the test, and captures screenshots after execution, which are then processed by the image understanding module; (2.2) The image understanding module understands the screenshot pages captured during the test, identifies and generates the control tree under the page, and relies on two different neural networks to extract the state feature vector and action feature vector, respectively, as the state space and action space of the reinforcement learning module; (2.3) Reinforcement learning module: The two feature vectors extracted by the image understanding module are combined with the state space and action space of the reinforcement learning module and input into the neural network. The neural network selects the next interactive action and executes the interactive action according to the ε-greedy strategy. The abnormal problems detected during the exploration process are collected and the corresponding test reports are generated. In addition, each state transition and its reward value under the reward function are stored as data for experience replay; (3) Present the generated test report to the user.

2. The cross-platform reinforcement learning traversal testing method for mobile applications based on image understanding according to claim 1 is characterized by: In the image understanding module, the feature extraction part uses Canny edge detection, OCR technology, CNN classification algorithm, and LSTM recurrent neural network to obtain the control tree, state feature vector, and action feature vector.

3. The cross-platform reinforcement learning traversal testing method for mobile applications based on image understanding according to claim 1 is characterized by: In the reinforcement learning module, Deep Q-Network is used to fit the Q function and select interactive actions, which are then executed according to the ε-greedy strategy. Experience replay and a curiosity-driven reward function are used to update the Deep Q-Network in real time during testing.

Images