Automatic interaction processing method and device for program graphical user interface

Abstract

The embodiments of the invention provide an automatic interaction processing method and device for a program graphical user interface. The method comprises the steps that obtaining the program graphical user interface to be subjected to interaction processing d in the program running process; inputting the program graphical user interface into a preset recognition model, and recognizing an interaction control contained in the program graphical user interface, text content on the interaction control and the position where the text content appears; and performing automatic interaction processingon the program graphical user interface according to the identification result. According to the embodiments of the invention, the interaction control contained in the program graphical user interface, the text content on the interaction control and the appearing position of the text content are automatically identified by adopting a machine learning technology, therefore, automatic interaction processing of the graphical user interface of the program is completed, the dynamic analysis process of the application program is not interrupted, and the problem of dynamic analysis stagnation causedby the fact that the interaction process of a program interface cannot be completed automatically in the prior art is solved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method and device for automatic interactive processing of a program graphical user interface. Background technique [0002] The Android system is currently the mainstream system in the market, and its market share is greater than 80%, far exceeding other mobile device operating systems. Behind the prosperity and development of the Android system, the rich and colorful Android applications are indispensable. However, while Android applications enrich people's lives, they also bring various security risks to users. Malware such as malicious chargeback software, privacy stealing software, fraudulent software, and phishing software threaten users' privacy and funds all the time. Safety. [0003] In order to ensure the safety of users, major security vendors, application store providers, and research institutions have launched corresponding Android application security analysis ...

AI Technical Summary

Problems solved by technology

For common interfaces, the controls on the interface can be identified through interface analysis. However, for some custom interfaces or text implemented with graphics, the controls that should be clicked on the interface cannot be identified through interface analysis, and thus cannot Simulate users to click, select and other operations, resulting in the automatic installation of the application and the inability to automate the interaction process of the program interface, which makes the dynamic analysis stagnant

[0006] Aiming at the above problems, an effective solution has not yet been proposed

Images