Application program interface acquisition method, device, computer equipment and storage medium

Abstract

The present application relates to a method, device, computer equipment and storage medium for obtaining the interface of an application. The method comprises: in response to the startup operation of the application, obtaining a click command stream file, the click command stream file containing multiple click commands, each click command containing a corresponding interface element image; according to the interface element image contained in the click command, determining a target area matching the interface element image from the interface of the application; performing a click operation on the target area, and performing an interface jump in response to the click operation. The use of this method can reduce the development difficulty of automated clicks and improve product development efficiency; by finding the location of the interface element based on image matching technology, the compatibility between different types of applications is improved; and the situation where the interface element may not exist but the erroneous operation of clicking is still performed is avoided.

Description

Technical Field

[0001] The present application relates to the field of information security technology, and in particular to a method, device, computer equipment and storage medium for obtaining an application program interface. Background Art

[0002] With the prosperity of the App (Application) market, a large number of companies have launched their own applications, which has promoted the development of mobile Internet. However, the quality of apps on the market varies greatly, and there are many apps with illegal content, such as pornography, terrorism, and politics. At present, the method of App interface image content recognition can be used to detect whether there are any content violations in the App.

[0003] In traditional technology, the interface images of each App during operation can be automatically obtained in the following ways: based on the coordinates of the interface elements obtained by reverse parsing the App; automatically click on the interface elements according to the coordinate points to complete the interface jump; and take screenshots to obtain the interface images. However, there are still some Apps, such as HTML5 (HyperText Markup Language 5, Hypertext 5.0) applications, which cannot complete the automatic click interface jump task through reverse parsing. Alternatively, pre-configure fixed coordinate points of the interface elements for each interface; when testing the interface content, perform automatic click operations according to the fixed coordinate points; and take screenshots to obtain the interface images. However, some Apps have some interface elements that appear randomly during operation, and it is impossible to complete the automatic click interface jump task by means of fixed coordinate points. Therefore, the solutions in traditional technology cannot automatically click on the interface of any App to complete the interface jump task, and thus cannot obtain the interface images of each interface. Summary of the invention

[0004] Based on this, it is necessary to provide an interface acquisition method, device, computer equipment and storage medium for an application program that can automatically click on the interface of any App to complete the interface jump task in order to solve the above technical problems.

[0005] In a first aspect, an embodiment of the present application discloses a method for obtaining an interface of an application program, the method comprising:

[0006] In response to a startup operation of the application, a click command stream file is obtained, wherein the click command stream file includes a plurality of click commands, and each click command includes a corresponding interface element image;

[0007] According to the interface element image contained in the click command, determining a target area matching the interface element image from the interface of the application program;

[0008] A click operation is performed on the target area, and an interface jump is performed in response to the click operation.

[0009] In one embodiment, the method of generating the click command stream includes:

[0010] Starting from the main interface of the application, obtaining interface element images collected from the main interface;

[0011] Generate a first click command according to the interface element picture of the main interface;

[0012] In response to a click operation on an interface element in the main interface, a second interface is displayed, and so on, to obtain a click command corresponding to each interface;

[0013] The obtained click commands are saved in the generation order of the click commands to generate the click command stream file.

[0014] In one embodiment, determining a target area matching the interface element image from the interface of the application program according to the interface element image included in the click command includes:

[0015] Starting from the main interface of the application, each time a new interface is displayed, the corresponding click command is obtained according to the generation order of the click command in the click command stream file until the click command is executed;

[0016] According to the current interface element image contained in the currently acquired click command, a target area matching the current interface element is determined from the current display interface of the application.

[0017] In one embodiment, determining a target area matching the current interface element from the current display interface of the application according to the current interface element image contained in the currently acquired click command includes:

[0018] Taking a screenshot of the current display interface of the application to obtain a current interface image;

[0019] A target area matching the current interface element image is determined from the current interface image through template matching.

[0020] In one embodiment, performing a click operation on the target area includes:

[0021] Obtaining the location coordinates of the target area;

[0022] Determine the center point of the target area according to the position coordinates of the target area;

[0023] A click operation is performed on the center point.

[0024] In one embodiment, the method further comprises:

[0025] Every time a new interface is displayed, a screenshot is taken of the displayed interface to obtain the corresponding interface image;

[0026] Identify each interface image to obtain the interface content in each interface image;

[0027] If it is determined that the interface content corresponding to any interface image contains illegal content, a test result of application violation is generated;

[0028] If it is determined that the interface content corresponding to each interface image does not contain the illegal content, a test result of application compliance is generated.

[0029] In a second aspect, an embodiment of the present application discloses a device for acquiring an interface of an application program, the device comprising:

[0030] A file acquisition module, used to acquire a click command stream file in response to a startup operation of the application, wherein the click command stream file includes a plurality of click commands, and each click command includes a corresponding interface element image;

[0031] A target area determination module, used to determine a target area matching the interface element image from the interface of the application program according to the interface element image contained in the click command;

[0032] The interface jump module is used to perform a click operation on the target area and perform an interface jump in response to the click operation.

[0033] In one embodiment, the device further comprises:

[0034] A picture acquisition module, used to acquire interface element pictures collected from the main interface of the application program;

[0035] A click command generating module is used to generate a first click command according to the interface element picture of the main interface; in response to the click operation on the interface element in the main interface, display a second interface, and so on, to obtain a click command corresponding to each interface;

[0036] The click command stream generating module is used to save the obtained click commands according to the generation order of the click commands and generate the click command stream file.

[0037] In a third aspect, an embodiment of the present application discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in any one of the embodiments of the first aspect are implemented.

[0038] In a fourth aspect, an embodiment of the present application discloses a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in any one of the embodiments of the first aspect above are implemented.

[0039] The interface acquisition method, device, computer equipment and storage medium of the above-mentioned application save the interface element image in advance to generate a click command stream file; after detecting the startup operation of the application, obtain the interface element image from the click command of the click command stream file; determine the matching target area from the current interface according to the obtained interface element image, and perform an automatic click operation on the target area to achieve interface jump. The technical solution of the present application is based on image matching technology to achieve automatic clicks. Compared with traditional technologies, there is no need to understand the internal structure, development framework, programming language and other technical details of the application, reverse parse the application, and record the fixed coordinate positions of the interface elements in advance to implement click operations on the application, which reduces the difficulty of developing automatic clicks and improves product development efficiency; by finding the location of the interface element based on image matching technology, the compatibility between different types of applications is improved; and it avoids the situation where the interface element may not exist but the erroneous operation of still performing a click is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 An application environment diagram of a method for obtaining an interface of an application program in one embodiment;

[0041] Figure 2 A schematic diagram of a flow chart of a method for obtaining an interface of an application program in an embodiment;

[0042] Figure 3 A schematic diagram of a flow chart of steps for generating a click command stream file in one embodiment;

[0043] Figure 4 A schematic flow chart of a step of determining a target area in another embodiment;

[0044] Figure 5 A schematic diagram of a flow chart of a method for obtaining an interface of an application program in an embodiment;

[0045] Figure 6 A schematic diagram of a flow chart of steps for generating a click command stream file in one embodiment;

[0046] Figure 7A structural block diagram of an interface acquisition device for an application program in one embodiment;

[0047] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The method for obtaining the interface of an application provided in the present application can be applied to a terminal. The terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The interface acquisition of the application is implemented based on an automated click program. The terminal is pre-deployed with an execution file of the automated click program. After automatically triggering the start operation of the application, obtain the click command stream file corresponding to the application. Obtain an interface element image from the click command in the click command stream file. Determine the target area in the interface that matches the interface element image based on the interface element image. Automatically perform a click operation on the target area, jump to the next interface, and then implement information security detection on the next interface.

[0050] In another embodiment, the method for obtaining the interface of the application provided in the present application can also be applied to Figure 1 In the application environment shown. Among them, the terminal 110 communicates with the server 120 through the network. The server 120 pre-stores the click command stream files of multiple applications for at least one terminal 110 to call. The terminal 110 is pre-deployed with an automatic click program. After the terminal 110 automatically triggers the start operation of the application, it sends a request to the server 120 to obtain the click command stream file, and the acquisition request carries the associated information of the application (for example, unique identification, application name, etc.), requests the server 120 to find the corresponding click command stream file according to the associated information of the application, and returns the click command stream file to the terminal 110. The terminal 110 obtains the interface element image of the click command from the click command stream file. According to the interface element image, the target area in the interface that matches the interface element image is determined. The click operation is performed on the target area, and the next interface is jumped to, and then the information security detection is performed on the next interface. Among them, the terminal 110 can be, but not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 120 can be implemented with an independent server or a server cluster composed of multiple servers.

[0051] In one embodiment, Figure 2As shown, a method for obtaining an interface of an application is provided, and the method is described by taking the method applied to a terminal as an example, and includes the following steps:

[0052] Step S210, in response to the start-up operation of the application, a click command stream file is obtained, the click command stream file includes a plurality of click commands, and each click command includes a corresponding interface element image.

[0053] Among them, an application refers to a computer program that is used to complete one or more specific tasks. It runs in user mode, can interact with users, and has a visual user interface.

[0054] The click command stream is a command combination consisting of click commands. These click commands are arranged in a certain order and can complete the automatic click function of the interface elements. The click commands in the click command stream can be arranged in the order of opening the application interface; or randomly arranged; or customized by the user according to needs. Each click command carries an interface element image.

[0055] Interface elements refer to elements contained in the application interface that can meet the user's interaction requirements, such as controls, hyperlinks, etc. Interface element images are images captured by users in advance by taking screenshots of the application interface.

[0056] Specifically, the information security detection of the application can be manually triggered by the user; it can also be automatically triggered by the terminal, for example, it is pre-configured to automatically perform information security detection on the application at a fixed time point every day. The terminal responds to the start instruction of the information security detection and performs a start operation on the application to be detected. The terminal obtains the associated information of the application to be detected. According to the associated information of the application, the corresponding click command stream file is obtained from the local database or server.

[0057] Step S220: determining a target area matching the interface element image from the interface of the application program according to the interface element image included in the click command.

[0058] Specifically, the terminal starts from the first click command, and each time it enters the interface of an application, it obtains the next click command from the click command stream file according to the arrangement of the click commands in the click command stream, and obtains the interface element image from the obtained click command. The terminal automatically matches the current interface element image with the current interface of the application, and determines the target area that best matches the current interface element image from the current interface of the application. Among them, automatic matching can be achieved through template matching, image feature retrieval and other technologies.

[0059] Step S230: performing a click operation on the target area, and performing an interface jump in response to the click operation.

[0060] Specifically, since the target area is the area that best matches the interface element image, it can be considered that the area contains clickable interface elements, and the clickable interface elements are consistent with the interface elements contained in the interface element image. The terminal automatically performs a click operation on the target area, and automatically responds to the click operation to jump to the page interface. Accordingly, the terminal obtains the next click command from the click command stream, and repeats steps S220 and S230, and so on, until all click commands in the click command stream are executed.

[0061] In a possible embodiment, there may be more than one application. An application list is predefined, and the applications included in the application list are applications that need to be tested for information security. The terminal executes the contents described in steps S210 to S230 for each application in sequence according to the order defined in the application list until all applications included in the application list are tested.

[0062] In one possible embodiment, the application can be installed in the terminal before the information security test; or it can be automatically downloaded and installed in the terminal according to the associated information of the application when the execution instruction of the information security test is triggered. After the information security test is completed, the terminal saves the test results and automatically uninstalls the tested application, thereby saving the operating space of the terminal device.

[0063] In one possible embodiment, the technical solution described in steps S210 to S230 can also be applied to simulate private configuration attributes such as inputting an account number and inputting a password. The above technical solution can be used to dynamically configure settings such as passwords (such as mobile phone passwords), thereby avoiding the situation where subsequent operations cannot be performed due to failure of the verification logic.

[0064] In the interface acquisition method of the above application, the interface element image is saved in advance to generate a click command stream file; after the startup operation of the application is detected, the interface element image is obtained from the click command of the click command stream file; according to the obtained interface element image, the matching target area is determined from the current interface, and an automatic click operation is performed on the target area to achieve interface jump. The technical solution of the present application is based on image matching technology to achieve automatic clicks. Compared with traditional technologies, there is no need to understand the internal structure, development framework, programming language and other technical details of the application, reverse parse the application, and record the fixed coordinate positions of the interface elements in advance to implement click operations on the application, which reduces the difficulty of developing automatic clicks and improves product development efficiency; by finding the location of the interface element based on image matching technology, the compatibility between different types of applications is improved; and it avoids the situation where the interface element may not exist but the erroneous operation of still performing a click is avoided.

[0065] In one embodiment, a method for generating a click command stream file is described. Figure 3 As shown, this can be achieved by following the steps below:

[0066] Step S310, starting from the main interface of the application, obtaining interface element images collected from the main interface.

[0067] Step S320, generating a first click command according to the interface element image of the main interface.

[0068] In this embodiment, the click command stream file is an ordered list, which is manually defined by the user. Each item in the list describes the interface element image that needs to be clicked at present. The entire click command stream describes a complete click process of the application.

[0069] Specifically, for any application that requires information security detection, the user triggers a startup operation on the application, so that the client of the application responds to the startup operation and displays the main interface of the application. Starting from the main interface of the application, the user performs a screenshot operation on the interface elements in the main interface, so that the terminal obtains the corresponding interface element screenshot. The user adds the interface element screenshot to the click command to generate the first click command.

[0070] Step S330, in response to a click operation on an interface element in the main interface, display a second interface, and so on, to obtain a click command corresponding to each interface.

[0071] Step S340, saving the obtained click commands according to the generation order of the click commands, and generating a click command stream file.

[0072] Specifically, after generating the first click command, the user triggers the interface element in the main interface, causing the terminal to display the second interface. Referring to step S310 and step S320, continue to obtain screenshots of the interface elements in the second interface, and generate corresponding click commands based on the obtained screenshots of the interface elements. The user refers to the above steps to open each interface of the application in turn, and takes real-time screenshots of the interface elements in each interface to obtain the corresponding click command until all interfaces of the application are accessed. The user saves the obtained click commands in the order in which the click commands are generated, generates a click command stream file corresponding to the application, and saves the obtained click command stream file in a local database or server.

[0073] In one possible embodiment, the interface of the application may include multiple interface elements. In this case, a click command including a "return interface element" image can be inserted into the click command stream file, so that the terminal can automatically perform click operations on different interface elements in the same interface, thereby obtaining the interface image of the application more comprehensively.

[0074] In this embodiment, the user manually opens each interface of the application in turn, takes real-time screenshots of the interface elements in each interface, thereby obtaining corresponding click commands, and generates a click command stream file according to the generation order of the click commands, so that the obtained click command stream file can cover the complete click process of the application, thereby helping to improve the accuracy of information security detection of the application.

[0075] In one embodiment, Figure 4 As shown, step S220, according to the interface element image included in the click command, determines the target area matching the interface element image from the interface of the application program, which can be achieved by the following steps:

[0076] Step S221, starting from the main interface of the application, each time a new interface is displayed, the corresponding click command is obtained according to the generation order of the click command in the click command stream file until the click command is executed.

[0077] Step S222: According to the current interface element image contained in the currently acquired click command, a target area matching the current interface element is determined from the current display interface of the application.

[0078] Specifically, the terminal automatically starts the application in response to the start instruction of the information security detection. The terminal starts the pre-deployed automatic click program, obtains the click command stream file corresponding to the application through the automatic click program, and parses the click command sequence from the click command stream file. The click command stream file used in this embodiment is based on Figure 3 The corresponding implementation example is obtained. Therefore, starting from the main interface of the application, after jumping to the current display interface, the corresponding current click command is obtained according to the order of the click commands in the click command flow, and the current interface element image is obtained from the current click command. The area that best matches the current interface element image is determined from the current display interface as the target area. The click operation is automatically performed on the target area, and the next display interface is continued to be entered, and the above execution process is repeated until all click commands are executed.

[0079] In this embodiment, by using a click command stream file that covers the complete click process of the application, each interface of the application can be obtained, which helps to improve the accuracy of information security detection of the application.

[0080] In one embodiment, the matching process of the interface element image and the application interface is described. Step S222, according to the current interface element image contained in the currently acquired click command, determines the target area matching the current interface element from the current display interface of the application, including: taking a screenshot of the current display interface of the application to obtain the current interface image; and determining the target area matching the current interface element image from the current interface image by template matching.

[0081] Specifically, after jumping to the current display interface of the application, the terminal automatically takes a screenshot of the current display interface to obtain the current interface image. Using the current interface element image as a reference, slide the current interface element image on the current interface image in a preset direction (for example, from left to right, from top to bottom) to obtain an image block of the same size as the current interface element image. Calculate the similarity between each image block and the current interface element image, and take the image block with the highest similarity as the target area. Among them, the similarity can be obtained using a matching algorithm (for example, square difference matching, standard square difference matching, correlation matching, etc.) or an algorithm such as image feature correlation.

[0082] In this embodiment, the click area is determined from the current interface image by using template matching technology. The matching process is intuitive, and because the template matching technology is simple and easy to debug, the debugging cost is reduced and the robustness of the automated click program is improved.

[0083] In one embodiment, step S230, performing a click operation on the target area, includes: obtaining the position coordinates of the target area; determining the center point of the target area according to the position coordinates of the target area; and performing a click operation on the center point.

[0084] Specifically, the shape of the image block obtained in the process of template matching can be any shape, such as a circle, a rectangle, etc. The terminal obtains the position coordinates of the target area, and calculates the position coordinates of the center point of the target area based on the position coordinates of the target area. Then, a click operation is performed on the position coordinates of the center point. Exemplarily, the position coordinates can be represented by pixel coordinates; the image block is a rectangle. After determining the target area, the terminal obtains the pixel coordinates of the four endpoints of the target area. The pixel coordinates of the center point of the rectangle are calculated based on the pixel coordinates of the four endpoints. Then, a click operation is performed on the pixel coordinates of the center point, so that the application automatically jumps to the next interface.

[0085] In this embodiment, after determining the target area that best matches the interface element image, the center point of the target area is further determined based on the position coordinates of the target area, and a click operation is performed on the center point, which can improve the accuracy of automatic clicking.

[0086] In one embodiment, in step S230, each time a new interface is displayed, a screenshot of the displayed interface is taken, and after obtaining the corresponding interface image, the following steps are also included: identifying each interface image to obtain the interface content in each interface image; if it is determined that there is illegal content in the interface content corresponding to any interface image, a test result of application violation is generated; if it is determined that there is no illegal content in the interface content corresponding to each interface image, a test result of application compliance is generated.

[0087] Specifically, each time an application opens an interface, a screenshot command is automatically triggered to obtain the corresponding interface image. For each interface image, the interface content in each interface image is identified by a pre-deployed deep learning algorithm, and based on the obtained interface content, it is determined whether the application has information security risks. Exemplarily, character recognition is performed on each interface image through OCR (Optical Character Recognition) technology to obtain the text content in each interface image. Sensitive words are identified for the obtained text content. If it is determined that sensitive words exist in the text content, it is considered that the application contains illegal content, and a test result of the application violation is generated. Furthermore, if it is determined that there is no illegal content in the interface content corresponding to each interface image, a test result of the application compliance is generated.

[0088] In this embodiment, the complete interface content of the application is obtained based on an automated click program, and the terminal determines whether the application contains any illegal content based on the complete interface content. No manual intervention by the user is required, and the degree of automation is high, thereby improving the efficiency of the test; by introducing deep learning technology, the accuracy of the test can be improved.

[0089] In one embodiment, a specific method for obtaining an interface of an application is provided, and an application under an Android operating system is used as an example for description. Figure 5 As shown, the following steps are included:

[0090] Step S510: the terminal automatically triggers the start-up operation of the application.

[0091] Step S520: The terminal obtains the click command stream file.

[0092] The click command stream file contains multiple click commands, each of which contains a corresponding interface element image. Multiple click commands are sorted according to the actual interface opening order of the application. Figure 6 , explain how to generate click command stream files:

[0093] Step S521, starting the application in response to a start operation triggered by the user.

[0094] Step S522, starting from the main interface of the application, in response to the user's screenshot operation on the interface element in the current interface, obtain the corresponding interface element image. The interface element is the element that needs to be clicked during the test process.

[0095] Step S523, generating a current click command according to the current interface element image, and adding the current click command to the click command stream.

[0096] Step S524, in response to the click operation on the current interface element, jump to the next interface. If there is an interface element image to be clicked in the next interface, repeat steps S511 to S514 until the last interface of the application is reached and all interface element images are obtained.

[0097] Step S525, saving the obtained click commands according to the generation order of the click commands (ie the order in which the application interfaces are opened) to generate a click command stream file.

[0098] Step S530, starting from the main interface of the application, each time a new interface is displayed, the corresponding click command is obtained according to the generation order of the click commands in the click command stream file. Further, if all click commands in the click command stream are executed, the process ends.

[0099] Step S540, taking a screenshot of the current display interface of the application to obtain a current interface image.

[0100] Specifically, the automatic click program captures the entire current screen content of the application program through an ADB (Andorid Debug Bridge) command as the current interface image.

[0101] Step S550, determining a target area matching the current interface element image from the current interface image through template matching.

[0102] Step S560: acquiring the position coordinates of the target area, and determining the center point of the target area according to the position coordinates of the target area.

[0103] Step S570: perform a click operation on the center point, and perform an interface jump in response to the click operation.

[0104] Furthermore, information security detection can be performed on the application based on the obtained interface images. Specifically, each time a new interface is displayed, a screenshot of the displayed interface is taken to obtain the corresponding interface image; each interface image is identified to obtain the interface content in each interface image; if it is determined that there is illegal content in the interface content corresponding to any interface image, a test result of application violation is generated; if it is determined that there is no illegal content in the interface content corresponding to each interface image, a test result of application compliance is generated.

[0105] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0106] In one embodiment, Figure 7 As shown, an application interface acquisition device 700 is provided, comprising: a file acquisition module 701, a target area determination module 702 and an interface jump module 703, wherein:

[0107] The file acquisition module 701 is used to obtain a click command stream file in response to the startup operation of the application, where the click command stream file includes multiple click commands, and each click command includes a corresponding interface element image; the target area determination module 702 is used to determine a target area that matches the interface element image from the interface of the application according to the interface element image included in the click command; the interface jump module 703 is used to perform a click operation on the target area and perform an interface jump in response to the click operation.

[0108] In one embodiment, the device 700 also includes: an image acquisition module, which is used to obtain interface element images collected from the main interface of the application program; a click command generation module, which is used to generate a first click command based on the interface element images of the main interface; in response to a click operation on an interface element in the main interface, a second interface is displayed, and so on, to obtain a click command corresponding to each interface; a click command stream generation module, which is used to save the obtained click commands in the order in which the click commands are generated, and to generate a click command stream file.

[0109] In one embodiment, the target area determination module 702 includes: a click command acquisition unit, which is used to start from the main interface of the application, and each time a new interface is displayed, the corresponding click command is obtained according to the generation order of the click command in the click command stream file until the click command is executed; a target area determination unit, which is used to determine the target area matching the current interface element from the current display interface of the application according to the current interface element image contained in the currently obtained click command.

[0110] In one embodiment, the target area determination unit includes: a screenshot subunit, which is used to take a screenshot of the current display interface of the application to obtain a current interface image; and a target area determination subunit, which is used to determine a target area that matches the current interface element image from the current interface image through template matching.

[0111] In one embodiment, the interface jump module 703 includes: a position acquisition unit, used to acquire the position coordinates of the target area; a center point determination unit, used to determine the center point of the target area according to the position coordinates of the target area; and a click unit, used to perform a click operation on the center point.

[0112] In one embodiment, the device 700 also includes: a screenshot module, which is used to take a screenshot of the displayed interface each time a new interface is displayed to obtain a corresponding interface image; an image recognition module, which is used to recognize each interface image to obtain the interface content in each interface image; an application testing module, which is used to generate a test result of application violation if it is determined that there is illegal content in the interface content corresponding to any interface image; and generate a test result of application compliance if it is determined that there is no illegal content in the interface content corresponding to each interface image.

[0113] For the specific definition of the interface acquisition device of the application program, please refer to the definition of the interface acquisition method of the application program in the above text, which will not be repeated here. Each module in the interface acquisition device of the above application program can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0114] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for obtaining an interface of an application program is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0115] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0117] In response to the startup operation of the application, a click command stream file is obtained, the click command stream file includes multiple click commands, and each click command includes a corresponding interface element image; based on the interface element image included in the click command, a target area matching the interface element image is determined from the interface of the application; a click operation is performed on the target area, and an interface jump is performed in response to the click operation.

[0118] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0119] Starting from the main interface of the application, obtain the interface element pictures collected from the main interface; generate the first click command based on the interface element pictures of the main interface; in response to the click operation on the interface element in the main interface, display the second interface, and so on, to obtain the click command corresponding to each interface; save the obtained click commands in the order in which the click commands are generated, and generate a click command stream file.

[0120] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0121] Starting from the main interface of the application, each time a new interface is displayed, the corresponding click command is obtained according to the generation order of the click commands in the click command stream file until the click command is executed; based on the current interface element image contained in the currently obtained click command, the target area that matches the current interface element is determined from the currently displayed interface of the application.

[0122] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0123] A screenshot of the current display interface of the application is taken to obtain the current interface image; a target area matching the current interface element image is determined from the current interface image through template matching.

[0124] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0125] Obtain the position coordinates of the target area; determine the center point of the target area according to the position coordinates of the target area; and perform a click operation on the center point.

[0126] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0127] Every time a new interface is displayed, a screenshot of the displayed interface is taken to obtain the corresponding interface image; each interface image is identified to obtain the interface content in each interface image; if it is determined that there is illegal content in the interface content corresponding to any interface image, a test result of application violation is generated; if it is determined that there is no illegal content in the interface content corresponding to each interface image, a test result of application compliance is generated.

[0128] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0129] In response to the startup operation of the application, a click command stream file is obtained, the click command stream file includes multiple click commands, and each click command includes a corresponding interface element image; based on the interface element image included in the click command, a target area matching the interface element image is determined from the interface of the application; a click operation is performed on the target area, and an interface jump is performed in response to the click operation.

[0130] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0131] Starting from the main interface of the application, obtain the interface element pictures collected from the main interface; generate the first click command based on the interface element pictures of the main interface; in response to the click operation on the interface element in the main interface, display the second interface, and so on, to obtain the click command corresponding to each interface; save the obtained click commands in the order in which the click commands are generated, and generate a click command stream file.

[0132] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0133] Starting from the main interface of the application, each time a new interface is displayed, the corresponding click command is obtained according to the generation order of the click commands in the click command stream file until the click command is executed; based on the current interface element image contained in the currently obtained click command, the target area that matches the current interface element is determined from the currently displayed interface of the application.

[0134] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0135] A screenshot of the current display interface of the application is taken to obtain the current interface image; a target area matching the current interface element image is determined from the current interface image through template matching.

[0136] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0137] Obtain the position coordinates of the target area; determine the center point of the target area according to the position coordinates of the target area; and perform a click operation on the center point.

[0138] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0139] Every time a new interface is displayed, a screenshot of the displayed interface is taken to obtain the corresponding interface image; each interface image is identified to obtain the interface content in each interface image; if it is determined that there is illegal content in the interface content corresponding to any interface image, a test result of application violation is generated; if it is determined that there is no illegal content in the interface content corresponding to each interface image, a test result of application compliance is generated.

[0140] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0141] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for obtaining an application interface, characterized in that: The method comprises: In response to a startup operation of the application, a click command stream file is obtained, wherein the click command stream file includes a plurality of click commands, and each click command includes a corresponding interface element image; The method for generating the click command stream file includes: Starting from the main interface of the application, obtaining interface element images collected from the main interface; Generate a first click command according to the interface element picture of the main interface; In response to a click operation on an interface element in the main interface, a second interface is displayed, and so on, to obtain a click command corresponding to each interface; Saving the obtained click commands according to the generation order of the click commands to generate the click command stream file; According to the interface element image contained in the click command, determining a target area matching the interface element image from the interface of the application program; Performing a click operation on the target area, and performing an interface jump in response to the click operation; Every time a new interface is displayed, a screenshot is taken of the displayed interface to obtain the corresponding interface image; Identify each interface image to obtain the interface content in each interface image; If it is determined that the interface content corresponding to any interface image contains illegal content, a test result of application violation is generated; If it is determined that the illegal content does not exist in the interface content corresponding to each interface image, a test result of application compliance is generated.

2. The method according to claim 1, characterized in that The step of determining a target area matching the interface element image from the interface of the application program according to the interface element image included in the click command includes: Starting from the main interface of the application, each time a new interface is displayed, the corresponding click command is obtained according to the generation order of the click command in the click command stream file until the click command is executed; According to the current interface element image contained in the currently acquired click command, a target area matching the current interface element is determined from the current display interface of the application.

3. The method according to claim 2, characterized in that The step of determining a target area matching the current interface element from the current display interface of the application program according to the current interface element image contained in the currently acquired click command includes: Taking a screenshot of the current display interface of the application to obtain a current interface image; A target area matching the current interface element image is determined from the current interface image through template matching.

4. The method according to claim 1, characterized in that: Performing a click operation on the target area includes: Obtaining the location coordinates of the target area; Determine the center point of the target area according to the position coordinates of the target area; A click operation is performed on the center point.

5. A device for acquiring an application interface, characterized in that: The device comprises: A file acquisition module, used to acquire a click command stream file in response to a startup operation of the application, wherein the click command stream file includes a plurality of click commands, and each click command includes a corresponding interface element image; The method for generating the click command stream file includes: Starting from the main interface of the application, obtaining interface element images collected from the main interface; Generate a first click command according to the interface element picture of the main interface; In response to a click operation on an interface element in the main interface, a second interface is displayed, and so on, to obtain a click command corresponding to each interface; Saving the obtained click commands according to the generation order of the click commands to generate the click command stream file; A target area determination module, used to determine a target area matching the interface element image from the interface of the application program according to the interface element image contained in the click command; An interface jump module, used to perform a click operation on the target area and perform an interface jump in response to the click operation; Every time a new interface is displayed, a screenshot is taken of the displayed interface to obtain the corresponding interface image; Identify each interface image to obtain the interface content in each interface image; If it is determined that the interface content corresponding to any interface image contains illegal content, a test result of application violation is generated; If it is determined that the illegal content does not exist in the interface content corresponding to each interface image, a test result of application compliance is generated.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images