Mobile application cross-platform reinforcement learning traversal test technology based on depth image understanding

Abstract

A mobile application cross-platform reinforcement learning traversal test technology based on depth image understanding comprises an interaction module, a depth image understanding module and a reinforcement learning module. And the interaction module performs screen capture on the operation state of the apk, provides the captured state to the depth image understanding module, and selects action execution through the reinforcement learning module, so as to interact with the mobile application. The depth image understanding module carries out screenshot and analysis on a current interface of an application through a screenshot encoder, and respectively generates state and feature vectors after executable action encoding. The reinforcement learning module analyzes the advantages and disadvantages of the state and action pairs through a DQN model, selects the optimal executable action, and achieves the efficient exploration of the mobile application state space.

Description

technical field [0001] The invention belongs to the field of software testing. Automated traversal testing is performed on the provided mobile applications. On the front end, information is extracted through image understanding of screenshots. On the bottom layer, deep reinforcement learning methods are used to update the exploration strategy and generate test inputs. Finally, the purpose of testing the quality of mobile applications and discovering vulnerabilities is achieved. Background technique [0002] With the rapid development of the Internet and electronic devices in recent years, people's dependence on the Internet has gradually increased. Mobile applications have been related to all aspects of social life. Under the background of the rapid development of the Internet, mobile applications on different platforms are rapidly updated and iterated. But in such a situation, the quality of mobile applications is difficult to be guaranteed, so the testing technology for m...

AI Technical Summary

Problems solved by technology

Our invention can solve the problem of page information capture by understanding the information contained in the captured screenshots when the mobile application is running, and adjusting the exploration strategy of the test accordingly

Images