Application program testing method and device, electronic equipment and storage medium

Abstract

The invention discloses an application program testing method and device, electronic equipment and a storage medium. The coverage rate and accuracy of application program automatic testing are improved. The method comprises the steps that after a target application program runs, page state information of a graphical user interface of the target application program is acquired; generating a page state description text corresponding to the page state information according to the natural language description template; the page state description text serves as a prompt text to be input into a large language model, operation indication information is obtained, and the large language model is used for predicting operation indication information of the next step based on the prompt text; and indicating the target application program to execute corresponding operation according to the operation indication information.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to an application testing method, device, electronic device and storage medium. Background Art

[0002] With the rapid growth of mobile applications (Application, APP), application automation testing has been more and more widely used. Application automation testing can be used to automatically test various mobile applications to capture simulated user interface interactions and detect application functions and performance. Application automation testing helps detect malicious applications and indicates potential malicious behavior and data leakage risks by analyzing the application's behavior and page status in real time.

[0003] In the related art, the automated testing methods for applications include testing methods based on random events and testing methods based on graphical user interface (GUI) models. The events generated by the testing methods based on random events are random and cannot accurately simulate the behavior patterns of real users, resulting in the inability to accurately simulate the actual operations of users. The testing method based on the GUI model is to build a GUI model of the application under test, use the built GUI model to explore the application and generate events, while the testing method based on the GUI model requires building different GUI models for different applications, and each application has hundreds or even thousands of different pages, each page contains different interaction methods and logics, and the testing method based on the GUI model cannot adapt to the ever-changing application pages, resulting in the inability to cover all possible interaction scenarios.

[0004] Therefore, how to improve the coverage and accuracy of application automation testing is one of the technical problems that need to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide an application testing method, device, electronic device, and storage medium, which improve the coverage and accuracy of application automated testing.

[0006] In a first aspect, an embodiment of the present application provides an application testing method, comprising:

[0007] When the target application is running, obtaining page status information of the graphical user interface of the target application;

[0008] Generate a page status description text corresponding to the page status information according to a natural language description template;

[0009] Inputting the page status description text as prompt text into a large language model to obtain operation instruction information, wherein the large language model is used to predict the next operation instruction information based on the prompt text;

[0010] Instruct the target application to perform a corresponding operation according to the operation instruction information.

[0011] In a possible implementation, the page status information of the graphical user interface includes component attribute information, historical operation information, and task description information in the graphical user interface, and the component attribute information includes component name information and component operation type information;

[0012] Generating a page status description text corresponding to the page status information according to the natural language description template specifically includes:

[0013] Determine, according to the correspondence between the component operation type information and the natural language description template, a target natural language description template corresponding to the component operation type information;

[0014] A page status description text corresponding to the page status information is generated according to the component attribute information, historical operation information, task description information and the target natural language description template in the graphical user interface.

[0015] In a possible implementation manner, the target natural language description template includes a return format type requirement, where the return format type requirement is used to indicate a format type of the operation instruction information returned by the large language model.

[0016] In a possible implementation, instructing the target application to perform a corresponding operation according to the operation instruction information specifically includes:

[0017] Generate corresponding graphical user interface operation instructions according to the operation instruction information;

[0018] The graphical user interface operation instruction is sent to the target application to instruct the target application to perform the operation corresponding to the graphical user interface operation instruction.

[0019] In a possible implementation, the target application is installed in a preset operating system simulator, and the preset operating system simulator is set in a sandbox.

[0020] In a second aspect, an embodiment of the present application provides an application testing device, including:

[0021] An acquisition module, used for acquiring page status information of a graphical user interface of the target application after the target application is running;

[0022] A generation module, used to generate a page status description text corresponding to the page status information according to a natural language description template;

[0023] A prediction module, used for inputting the page status description text as a prompt text into a large language model to obtain operation instruction information, wherein the large language model is used for predicting the next operation instruction information based on the prompt text;

[0024] The instruction module is used to instruct the target application to perform a corresponding operation according to the operation instruction information.

[0025] In a possible implementation, the page status information of the graphical user interface includes component attribute information, historical operation information, and task description information in the graphical user interface, and the component attribute information includes component name information and component operation type information;

[0026] The generation module is specifically used to determine the target natural language description template corresponding to the operation type information of the component based on the correspondence between the operation type information of the component and the natural language description template; and to generate the page status description text corresponding to the page status information based on the component attribute information, historical operation information, task description information in the graphical user interface and the target natural language description template.

[0027] In a possible implementation manner, the target natural language description template includes a return format type requirement, where the return format type requirement is used to indicate a format type of the operation instruction information returned by the large language model.

[0028] In a possible implementation, the indication module is specifically used to generate a corresponding graphical user interface operation instruction according to the operation indication information; and send the graphical user interface operation instruction to the target application to instruct the target application to perform the operation corresponding to the graphical user interface operation instruction.

[0029] In a possible implementation, the target application is installed in a preset operating system simulator, and the preset operating system simulator is set in a sandbox.

[0030] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the application testing method described in the present application when executing the program.

[0031] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the application testing method described in the present application.

[0032] The beneficial effects of this application are as follows:

[0033] The application testing method, device, electronic device and storage medium provided by the embodiment of the present application, when the target application is running, the server obtains the page status information of the graphical user interface of the target application, generates the page status description text corresponding to the page status information according to the natural language description template; the page status description text is input into the large language model as a prompt text to obtain the operation instruction information, wherein the large language model is used to predict the next operation instruction information based on the prompt text, and then instruct the target application to perform the corresponding operation according to the operation instruction information. Compared with the prior art, the application testing method provided by the present application is applicable to various different applications. After the target application to be tested is running, the page status information of the graphical user interface of the target application is extracted, and converted into the page status description text according to the preset natural language description template, and the page status description text is used as the natural language prompt text to describe the page status of the current interactive interface, and the next operation instruction information is predicted based on the prompt text and the large language model to instruct the target application to perform the operation corresponding to the operation instruction information, while improving the coverage of the application automation test, the accuracy of the test is improved, and there is no need to pre-build GUI models for different applications, which improves the test efficiency.

[0034] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] Figure 1 A schematic diagram of an implementation flow of an application testing method provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of an implementation flow of generating a page status description text corresponding to page status information provided in an embodiment of the present application;

[0038] Figure 3 An example diagram of a natural language description template corresponding to an operation type of "selection operation" provided in an embodiment of the present application;

[0039] Figure 4An example diagram of a natural language description template corresponding to an operation type of "edit operation" provided in an embodiment of the present application;

[0040] Figure 5 An example diagram of converting component attribute information of a key component in a current graphical user interface into a description text of the component attribute information provided by an embodiment of the present application;

[0041] Figure 6 A schematic diagram of a graphical user interface of a target application and page status description text corresponding to page status information of the graphical user interface provided in an embodiment of the present application;

[0042] Figure 7 A schematic diagram of a description text of component attribute information of a text editing component provided in an embodiment of the present application;

[0043] Figure 8 An example diagram of the generation of a complete page status description text and the operation instructions returned by the large language model provided in an embodiment of the present application;

[0044] Fig. 9 A schematic diagram of an implementation flow of instructing a target application to perform a corresponding operation provided in an embodiment of the present application;

[0045] Fig.10 A schematic diagram of the structure of an application program testing device provided in an embodiment of the present application;

[0046] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to solve the problems in the background technology, the embodiments of the present application provide an application testing method, device, electronic device and storage medium.

[0048] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0049] In this article, it is necessary to understand that the technical terms involved in this application are:

[0050] 1. Sandbox: It is a mechanism for running programs safely, and it is often used to execute untrusted programs. According to certain security policies, sandboxes achieve isolation by strictly limiting the use of system resources by untrusted programs. The impact of malicious code in untrusted programs on the system will be limited to the sandbox and will not affect other parts of the system. In the field of malware analysis, dynamic analysis is often performed in a sandbox environment. By monitoring the network communication, file operations and other behaviors of the sample in the sandbox environment, useful feature information is collected to complete the maliciousness judgment and behavior analysis of the sample.

[0051] 2. Graphical User Interface: The graphical user interface of the operating system. The GUI system is one of the most important and complex systems of the operating system. It includes the following parts: window and graphics system, display synthesis system, user input system, and application framework system.

[0052] 3. Components: All UI (User Interface) elements displayed on the screen are called components, also known as controls and views.

[0053] 4. Android Debug Bridge (ADB): It is a debugging tool in Android SDK (Android Software development kit). ADB can directly operate and manage Android emulator or real Android device. Its functions include: running Shell (command line) of Android emulator or Android device, managing port mapping of Android emulator or Android device, uploading / downloading files of Android emulator or Android device, installing local APK (Android application package) to Android emulator or Android device, etc.

[0054] 5. Large Language Model (LLM): It is an artificial intelligence model designed to understand and generate human natural language. It is trained on a large amount of text data (prompts) and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, dialogue, etc. LLMs are characterized by their large scale and billions of parameters, which help them learn complex patterns in language data. Currently, the following large language models are widely used:

[0055] (1) GPT-4 (OpenAI) model: The GPT-4 model is one of the most famous large language models, with 1.8 trillion parameters. The model has shown remarkable performance in text generation, translation, and other tasks.

[0056] (2) ChatGLM2-6B (Tsinghua) model: The ChatGLM2-6B model is an open source, bilingual conversational language model that supports Chinese and English. It has 6.2 billion parameters and is based on the General Language Model (GLM) architecture. GLM is a general language model that is pre-trained with an autoregressive fill-in-the-blank objective and can be fine-tuned for various natural language understanding and generation tasks.

[0057] (3) LLaMA (Facebook) model: The LLaMA model is a set of basic language models with parameters ranging from 700 million to 6.5 billion. Among them, the LLaMA-13B model outperforms the GPT-3 model in most benchmarks. The model is trained on trillions of characters (tokens), and the results show that large model training can be completed using public datasets.

[0058] The application testing method provided in the embodiment of the present application can be applied to a server, a sandbox is provided in the server, a preset operating system simulator is provided in the sandbox, and the target application to be tested is installed in the preset operating system simulator. The preset operating system simulator can be, but is not limited to, an Android simulator, a Linux simulator, a Windows simulator, or any other operating system simulator. The embodiment of the present application does not limit this. The present application only uses the Android simulator as an example for illustration. The server can be an independent physical server or a cloud server that provides basic cloud computing services such as cloud servers, cloud databases, and cloud storage. The embodiment of the present application does not limit this.

[0059] Based on the above application scenarios, the following will refer to the attached Figures 1 to 9 The exemplary embodiments of the present application are described in more detail. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the implementation of the present application is not limited in any way. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0060] like Figure 1 As shown, it is a schematic diagram of the implementation process of the application testing method provided in the embodiment of the present application, which may specifically include the following steps:

[0061] S11. After the target application is running, obtain page status information of the graphical user interface of the target application.

[0062] During the specific implementation, an Android emulator is installed in the sandbox environment set up by the server in advance, and the installation command is executed through the ADB debugging tool in the Android SDK to install the target application package (APK) to be tested into the Android emulator. After the installation of the target application is completed, ADB is used to send the application running instruction to make the target application run in the foreground.

[0063] When the target application is running, the server extracts the page status information of the current graphical user interface of the target application through a script, wherein the page status information of the graphical user interface includes: component attribute information, historical operation information and task description information in the graphical user interface. The component attribute information may include the following information of the component: component name information and component operation type information, wherein the component operation type includes selection operation and edit operation, and the selection operation mainly includes: click, scroll, check and other operations, and corresponding operation identifiers (action id) may be set for different operation types of different components, and the component attribute information may also include the operation identifier corresponding to the component operation type. The server may extract the component attribute information of all components in the current graphical user interface of the target application, and may also extract the component attribute information of key components, which is not limited in the embodiment of the present application. If the component attribute information of the key component is extracted, after the server obtains all components of the current graphical user interface, it may search for the key components in the components of the current graphical user interface from the pre-set key component list, and then extract the component attribute information of the key components of the current graphical user interface from the current graphical user interface.

[0064] The historical operation information refers to the historical operations automatically performed by the target application before the current graphical user interface. The historical operation information is obtained to avoid repeated operations. The server can store the historical operation information of each graphical user interface of the target application in a database, and extract the historical operation information from the database for the current graphical user interface of the target application.

[0065] The task description information is a text description of the target application test, which can be pre-set, such as "You are an automated testing program for Android software, what you have to do is to test the functionality of the software as completely as possible." During implementation, it can be set according to needs. The embodiment of the present application is not limited to this. Using the task description information as prompt information of the large language model can enable the large language model to better understand the purpose of the input.

[0066] S12. Generate a page status description text corresponding to the page status information according to the natural language description template.

[0067] In implementation, the following Figure 2 The process shown generates a page status description text corresponding to the page status information, including the following steps:

[0068] S21. Determine a target natural language description template corresponding to the component operation type information according to the correspondence between the component operation type information and the natural language description template.

[0069] During specific implementation, the operation and maintenance personnel pre-set the corresponding natural language description template according to the operation type of the component. The server stores the correspondence between the operation type information and the natural language description template. The natural language description template includes: task description, component attribute information description, historical operation and return format type requirements. The return format type requirement is used to indicate the format type of the operation instruction information returned by the large language model. The natural language description template may also be called a prompt text template (also known as a Prompt text template), which may include a natural language description template corresponding to the operation type of "select operation" and a natural language description template corresponding to the operation type of "edit operation".

[0070] like Figure 3 As shown, it is an example diagram of a natural language description template corresponding to an operation type of "select operation", and the specific description text of the task description in the Prompt text template is "You are an automated testing program for Android software, what you have to do is to demonstrate the functionality of the software as completely as possible."

[0071] The specific description text of the component property information description is "The current state has the following UI views and corresponding actions, with action id: (The current state has the following user interface components and corresponding operations, operation ID:)", where the description text of the component property information of the current graphical user interface of the target application is embedded.

[0072] The specific description text of the historical operation is "I have already completed the following steps, which should not be performed again: (I have already completed the following steps, which should not be performed again:)", where the historical operation information obtained from the database is embedded.

[0073] The description text required by the return format type is "What action should I choose next? Just return the action id and nothing else.", which instructs the large language model to return the next action instruction information and only needs to return the action id.

[0074] like Figure 4 As shown in the figure, it is an example diagram of a natural language description template corresponding to the operation type "edit operation". The difference between it and the natural language description template corresponding to the "select operation" is only that the return format type requirement is different. The return format type requirement of the natural language description template is "What text should I enter to the viewwith text<text content> ? Just return the text and nothing else.(I should return the<text content> What text do you enter in the component? Just return the text)", which instructs the large language model to return the text to the edit box<text content> The text entered in .

[0075] In the embodiment of the present application, since different natural language description templates are set according to the operation type of the component, for text input, the present application can generate a text input prompt related to the current graphical user interface based on the natural language description text corresponding to the "editing operation", thereby solving the problem in the prior art that text input requires real-time judgment based on the current user interface, which makes operation difficult.

[0076] After the server obtains the page status information of the current graphical user interface of the target application, it determines the target natural language description template corresponding to the component operation type information in the obtained page status information according to the correspondence between the component operation type information and the natural language description template.

[0077] S22. Generate a page status description text corresponding to the page status information according to the component attribute information, historical operation information, task description information and the target natural language description template in the graphical user interface.

[0078] In specific implementation, after the server obtains the component attribute information of each key component of the current graphical user interface of the target application, it generates a description text of the component attribute information of the corresponding component according to the component attribute information of each key component.

[0079] Here is an example to illustrate: Figure 5As shown in FIG. 1 , it is an example diagram of converting component attribute information of key components in the current graphical user interface into description text of component attribute information. For each component, the description text of the component attribute information can be generated according to the component name information, the component operation type information and the operation identifier: “a view <name>that can <action>(action id)", that is, component <component name information> can <operation type information> (action id). For example, Figure 5 The component attribute information of the component "Search contacts" (query address book) at the top of the current graphical user interface includes: component name information "Search contacts", operation type information "click" and operation identifier "0" corresponding to the operation type information "click", then the description text of the component attribute information of the component "Search contacts" can be set to: "a view with text "Search contacts" that can click(0)". The component attribute information of the component "Phone contacts" (phone address book) below the component "Searchcontacts" includes: component name information "Phone contacts", operation type information "click" and operation identifier "5" corresponding to the operation type information "click", then the description text of the component attribute information of the component "Phone contacts" can be set to: "a view with text "Phone contacts" that can click(5)". The component property information of the component with the "+" icon at the lower right corner of the current graphical user interface includes: component name information "Create contacts", operation type information "click" and operation identifier "8" corresponding to the operation type information "click". The description text of the component property information of the component "+" can be set to "a view with "Create contacts" that can click(8)". The description text of the component property information of each key component in the current graphical user interface of the target application is generated in sequence according to the above method.

[0080] Furthermore, the server generates a page status description text corresponding to the page status information according to the description text of the component attribute information of the key components in the graphical user interface, the historical operation information, the task description information and the target natural language description template.

[0081] Specifically, the server embeds the descriptive text of the component attribute information, historical operation information, and task description information of each key component of the current graphical user interface of the target application into the corresponding position of the target natural language description template, and generates page status description text corresponding to the page status information of the current graphical user interface of the target application, thereby converting the page status information into natural language description text.

[0082] For example, if the target application previously executed Figure 5 After clicking the "+" of the "Create contacts" component in the target application, the target application's graphical user interface and the page status description text corresponding to the page status information of the graphical user interface are as follows: Figure 6 shown.

[0083] If the next step of the target application is to click on the "First name" component, the "First name" component includes two types of operations: edit and click, and the next step of clicking on the "First name" component is to edit, then the description text of the component attribute information for the text editing component will be generated as follows: Figure 7 The prompt sentence shown is: "What text should I enter to the view with text<First name> ? Just return thetext and nothing else.(I should return the<First name> What text is entered into the component? Just return the text)".

[0084] S13: Input the page status description text as prompt text into the large language model to obtain operation instruction information.

[0085] During specific implementation, the server can collect artificial test data of various commonly used applications to pre-train the large language model. Commonly used applications may include: social tool applications (such as various instant messaging applications), address book applications, notepad applications, tool applications (such as compass applications, calculator applications, timer applications), etc., and may also include any other type of applications, which is not limited in this embodiment of the present application. The server uses the page status description text corresponding to the page status information of each graphical user interface component of the application as the prompt text as the training sample large language model, and marks the operation that the application needs to perform next, so that the large language model learns the next operation instruction information, that is: the operation that needs to be performed next, adjusts the parameters of the large language model until the model converges, and obtains a pre-trained large language model. The large language model is used to predict the next operation instruction information based on the prompt text. Among them, the large language model can adopt any one of the GPT-4 model, ChatGLM2-6B model and LLaMA model, and can also adopt any other large language model, which is not limited in this embodiment of the present application.

[0086] In this step, the server inputs the page status description text corresponding to the page status information of the current graphical user interface of the target application into the pre-trained large language model to obtain operation instruction information.

[0087] like Figure 8 As shown, it is an example diagram of the generation of the page status description text corresponding to the complete page status information and the operation instructions returned by the large language model. Figure 8 It can be seen that the decision result of the large language model is: 8, which is the operation identifier corresponding to the operation type "click" of the component "Create contacts". That is, the next operation instruction information is: instructing the target application to perform the "click" component "Create contacts" operation. After the target application performs the operation, the graphical user interface displayed is as follows Figure 8 Shown on the right.

[0088] S14. Instruct the target application to perform a corresponding operation according to the operation instruction information.

[0089] In implementation, the following Fig. 9 The process shown instructs the target application to perform the corresponding operation, including the following steps:

[0090] S31. Generate corresponding graphical user interface operation instructions according to the operation instruction information.

[0091] During specific implementation, the server generates corresponding graphical user interface operation instructions according to the operation instruction information. The graphical user interface operation instructions may be GUI events corresponding to the operation instruction information.

[0092] S32: Send a graphical user interface operation instruction to the target application to instruct the target application to perform an operation corresponding to the graphical user interface operation instruction.

[0093] In specific implementation, the server can send a graphical user interface operation instruction to the target application through the ADB debugging tool. After receiving the graphical user interface operation instruction, the target application executes the operation corresponding to the graphical user interface operation instruction. Figure 8 In the example, after the big language outputs the operation instruction information, the server generates a graphical user interface operation instruction for clicking the component "Create contacts" (that is, a GUI event for clicking the component "Create contacts") according to the operation instruction information, and sends the GUI event for clicking the component "Create contacts" to the target application through the ADB debugging tool. After receiving the GUI event, the target application executes the operation of clicking the component "Create contacts".

[0094] The above process is a complete process of GUI analysis and operation. After completing the analysis and automated operation of a page, a new round of the above process will be started based on the changes in the page status of the current GUI, so as to perform complete functional automated testing of the target application.

[0095] The application testing method provided by the embodiment of the present application, when the target application is running, the server obtains the page status information of the graphical user interface of the target application, generates the page status description text corresponding to the page status information according to the natural language description template; the page status description text is input into the large language model as a prompt text to obtain the operation instruction information, wherein the large language model is used to predict the next operation instruction information based on the prompt text, and then instruct the target application to perform the corresponding operation according to the operation instruction information. Compared with the prior art, the application testing method provided by the present application is applicable to various different applications. After the target application to be tested is running, the page status information of the graphical user interface of the target application is extracted, and converted into the page status description text according to the preset natural language description template, and the page status description text is used as the natural language prompt text to describe the page status of the current interactive interface, and the next operation instruction information is predicted based on the prompt text and the large language model to instruct the target application to perform the operation corresponding to the operation instruction information, while improving the coverage of the application automation test, the accuracy of the test is improved, and there is no need to pre-build GUI models for different applications, which improves the test efficiency.

[0096] Based on the same inventive concept, an embodiment of the present application also provides an application testing device. Since the principle of solving the problem by the above-mentioned application testing device is similar to that of the above-mentioned application testing method, the implementation of the above-mentioned device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0097] like Fig.10 As shown, it is a schematic diagram of the structure of the application testing device provided in an embodiment of the present application, which may include:

[0098] The acquisition module 41 is used to acquire page status information of the graphical user interface of the target application after the target application is running;

[0099] A generating module 42, configured to generate a page status description text corresponding to the page status information according to a natural language description template;

[0100] A prediction module 43, used for inputting the page status description text as a prompt text into a large language model to obtain operation instruction information, wherein the large language model is used for predicting the next operation instruction information based on the prompt text;

[0101] The instruction module 44 is used to instruct the target application to perform a corresponding operation according to the operation instruction information.

[0102] In a possible implementation, the page status information of the graphical user interface includes component attribute information, historical operation information, and task description information in the graphical user interface, and the component attribute information includes component name information and component operation type information;

[0103] The generation module 42 is specifically used to determine the target natural language description template corresponding to the operation type information of the component based on the correspondence between the operation type information of the component and the natural language description template; and to generate the page status description text corresponding to the page status information based on the component attribute information, historical operation information, task description information in the graphical user interface and the target natural language description template.

[0104] In a possible implementation manner, the target natural language description template includes a return format type requirement, where the return format type requirement is used to indicate a format type of the operation instruction information returned by the large language model.

[0105] In a possible implementation, the indication module 44 is specifically configured to generate a corresponding graphical user interface operation instruction according to the operation indication information; and send the graphical user interface operation instruction to the target application to instruct the target application to perform the operation corresponding to the operation indication information.

[0106] In a possible implementation, the target application is installed in a preset operating system simulator, and the preset operating system simulator is set in a sandbox.

[0107] Based on the same technical concept, the embodiment of the present application also provides an electronic device 500, referring to Fig.11 As shown, the electronic device 500 is used to implement the application testing method described in the above method embodiment. The electronic device 500 of this embodiment may include: a memory 501, a processor 502, and a computer program stored in the memory and executable on the processor, such as an application testing program. When the processor executes the computer program, the steps in the above application testing method embodiments are implemented.

[0108] The specific connection medium between the memory 501 and the processor 502 is not limited in the embodiment of the present application. Fig.11 In the embodiment, the memory 501 and the processor 502 are connected via a bus 503. The bus 503 is connected to the processor 502 via a bus 503. Fig.11 The connection between other components is shown by bold lines, which is only for schematic illustration and is not intended to be limiting. The bus 503 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0109] The memory 501 may be a volatile memory, such as a random-access memory (RAM); the memory 501 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 501 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 501 may be a combination of the above memories.

[0110] Processor 502 is used to implement the application testing method provided in the embodiment of the present application.

[0111] An embodiment of the present application also provides a computer-readable storage medium that stores computer-executable instructions required to execute the above-mentioned processor, which includes a program required to execute the above-mentioned processor.

[0112] In some possible implementations, various aspects of the application testing method provided by the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the application testing method according to various exemplary implementations of the present application described above in this specification.

[0113] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0114] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0117] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0118] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.< / action> < / name>

Claims

1. An application testing method, characterized in that: include: When the target application is running, obtaining page status information of the graphical user interface of the target application; Generate a page status description text corresponding to the page status information according to a natural language description template; Inputting the page status description text as prompt text into a large language model to obtain operation instruction information, wherein the large language model is used to predict the next operation instruction information based on the prompt text; Instruct the target application to perform a corresponding operation according to the operation instruction information.

2. The method according to claim 1, characterized in that The page status information of the graphical user interface includes component attribute information, historical operation information, and task description information in the graphical user interface, and the component attribute information includes component name information and component operation type information; Generating a page status description text corresponding to the page status information according to the natural language description template specifically includes: Determine, according to the correspondence between the component operation type information and the natural language description template, a target natural language description template corresponding to the component operation type information; A page status description text corresponding to the page status information is generated according to the component attribute information, historical operation information, task description information and the target natural language description template in the graphical user interface.

3. The method according to claim 2, characterized in that The target natural language description template includes a return format type requirement, and the return format type requirement is used to indicate the format type of the operation instruction information returned by the large language model.

4. The method according to claim 1, characterized in that Instructing the target application to perform a corresponding operation according to the operation instruction information specifically includes: Generate corresponding graphical user interface operation instructions according to the operation instruction information; The graphical user interface operation instruction is sent to the target application to instruct the target application to perform the operation corresponding to the graphical user interface operation instruction.

5. The method according to claim 1, characterized in that The target application is installed in a preset operating system simulator, and the preset operating system simulator is set in a sandbox.

6. An application testing device, characterized in that: include: An acquisition module, used for acquiring page status information of a graphical user interface of the target application after the target application is running; A generation module, used to generate a page status description text corresponding to the page status information according to a natural language description template; A prediction module, used for inputting the page status description text as a prompt text into a large language model to obtain operation instruction information, wherein the large language model is used for predicting the next operation instruction information based on the prompt text; The instruction module is used to instruct the target application to perform a corresponding operation according to the operation instruction information.

7. The device according to claim 6, characterized in that The page status information of the graphical user interface includes component attribute information, historical operation information, and task description information in the graphical user interface, and the component attribute information includes component name information and component operation type information; The generation module is specifically used to determine the target natural language description template corresponding to the operation type information of the component based on the correspondence between the operation type information of the component and the natural language description template; and to generate the page status description text corresponding to the page status information based on the component attribute information, historical operation information, task description information in the graphical user interface and the target natural language description template.

8. The device according to claim 7, characterized in that The target natural language description template includes a return format type requirement, and the return format type requirement is used to indicate the format type of the operation instruction information returned by the large language model.

9. The device according to claim 6, characterized in that The instruction module is specifically used to generate a corresponding graphical user interface operation instruction according to the operation instruction information; and send the graphical user interface operation instruction to the target application to instruct the target application to perform the operation corresponding to the operation instruction information.

10. The device according to claim 6, characterized in that The target application is installed in a preset operating system simulator, and the preset operating system simulator is set in a sandbox.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the application program testing method according to any one of claims 1 to 5 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the application program testing method according to any one of claims 1 to 5 are implemented.

Images