A scene-oriented mobile application model fusion method

Abstract

The application discloses a scene-oriented mobile application model fusion method, which mainly comprises five parts of model construction, interface similarity calculation, jump similarity calculation, model traversal and scene generalization, and model fusion, and the specific steps of the method are as follows: user provides execution data obtained by exploring a mobile application, the execution data is constructed into an application execution path model, and function scene information is extracted; the model is traversed, and a corresponding matching path set is found based on input function scene path information; the interface similarity calculation algorithm and the jump similarity calculation algorithm are used for matching in the traversal process; finally, the two input execution path models are combined, and the matching function scene path information is marked on the combined model; the method is used for supporting downstream tasks of developers and testers, and can obtain accurate and more information-rich mobile application execution models at a lower cost.

Description

Technical Field

[0001] This invention relates to the field of software engineering, specifically to the interface and interface control information of mobile applications, the model construction technology of mobile applications, and the XML document similarity technology. In particular, it designs a scenario-oriented execution model fusion method for Android applications. Background Technology

[0002] In today's era, mobile applications are developing rapidly, with their scale and complexity constantly increasing. Unlike traditional command-line applications, a key characteristic of mobile applications is their event-driven model. When users interact with the interface of a mobile application, different interactive actions (such as clicking, long-pressing, swiping, etc.) trigger different events. After capturing these events, the mobile application executes corresponding processing actions, changes the application's state, and thus changes the displayed interface. Users then trigger events again on the new interface to interact, and this cycle constitutes the basic operating scenario of a mobile application. The event-triggered transitions between different states of a mobile application constitute the application's control flow.

[0003] In the software engineering development lifecycle, the model is a crucial component, and its quality has a vital impact on software development. The event-driven nature of mobile applications makes their execution paths more complex and variable than those of general programs. The transitions between application states triggered by different interaction events form a complex graph structure, which correspondingly increases the difficulty of modeling mobile applications.

[0004] When modeling mobile applications, common methods include manual exploration to trigger as many event transitions as possible, or using automated testing tools such as Monkey and UI Automator. Different testers and different testing tools will produce different traversal paths and graphs when modeling the same mobile application, resulting in different mobile application models. Models developed manually by testers are often incomplete but more consistent with the user's application logic; models generated by testing frameworks often have higher coverage but more chaotic execution sequences.

[0005] As mobile application development gradually becomes the mainstream of modern software development, the importance of mobile application models is becoming increasingly prominent. Currently, some works have established model databases for mobile applications, but manually building such databases is too costly, while machine learning models provide insufficient information. This invention proposes a scenario-oriented mobile application model fusion method. This method merges manually built models with scenario-specific functional information with machine-generated models, and generalizes the scenario-specific functional information to the merged model. By merging models, a large number of labeled models can be generalized from a small number of labeled models to a large number of unlabeled models, thus constructing a mobile application model library with ample information and a certain scale at a relatively low cost, facilitating subsequent research and development. Summary of the Invention

[0006] This invention provides a scenario-oriented mobile application model fusion method. The purpose is to use interface matching technology based on XML document similarity and jump matching technology based on component attribute equivalence to generalize manually annotated functional scenario path information to the execution path model of automated exploration. At the same time, the execution model of manual exploration and the execution model of automated exploration are fused to obtain a more comprehensive execution model with more functional scenario information, thereby assisting mobile application development and testing personnel in performing more downstream tasks.

[0007] To solve the above problems, the present invention is achieved through the following technical solution:

[0008] A scenario-oriented mobile application model fusion method, characterized by the following steps:

[0009] Before the execution of the method described in step 1), a mobile application modeling specification is predetermined, called the App Runtime Path. Users need to provide mobile application execution path data obtained through automated and manual exploration, including screenshots and layouts of the interface before and after each execution, as well as the specific operations of each execution, in order to construct the mobile application execution path model. Users also need to provide functional scenario information based on their understanding of the application, that is, an execution sequence that represents a specific functional scenario of the mobile application.

[0010] Step 2) Based on the two application execution path models and scene annotation information obtained in Step 1), a breadth-first search algorithm is used to traverse the automated exploration model. After traversal, several subgraphs matching the input functional scene paths are obtained. Then, depth-first search is used to convert the subgraphs into a set of paths.

[0011] Step 3) Merge the two application execution path models obtained in Step 1), and at the same time, label the corresponding functional scenario information onto the merged model according to the path set obtained in Step 2), thus obtaining a merged model with more functional scenario information.

[0012] The above-mentioned scenario-oriented mobile application model fusion method is characterized in that, in step 1), constructing the execution path model specifically includes the following steps:

[0013] Step 11) Input the application execution information for manual and automated exploration. The application execution information includes the interface information and jump information during execution. The interface information includes the interface screenshot and interface layout. The jump information includes the jump start state, target state and event.

[0014] Step 12) Input the application's functional scenario information;

[0015] Step 13) Construct an execution path model for the application based on the information from Step 11). The execution path model includes a directed graph with states as nodes and jump relationships as edges, and the application's metadata.

[0016] Step 14) Construct functional scenario information based on the information from Steps 11) to 13). The functional scenario information is a sequence of jumps in the application execution path. Each adjacent jump in the sequence is connected end to end to form a path representing a specific function of the application.

[0017] The above-mentioned scenario-oriented mobile application model fusion method is characterized in that step 2) of traversing the execution path model specifically includes the following steps:

[0018] Step 21) Use an XML layout-based state similarity algorithm, use tree edit distance to measure the similarity between interface layout structures, use character edit distance and histogram distance to estimate the lower bound of tree edit distance, and use the normalized estimated value as the standard for similarity evaluation;

[0019] Step 22) Use a jump similarity algorithm based on component attribute equivalence and component area overlap rate to determine the degree of jump similarity based on whether the attributes of the components associated with the jump event are equivalent, whether the corresponding areas of the components overlap, and whether the jump event types are the same.

[0020] Step 23) Use a breadth-first search-based traversal algorithm to find a subgraph in the execution path model that matches the input scene path;

[0021] Step 24) Use the depth-first search algorithm to convert the subgraph obtained in step 23) into a set of scene paths.

[0022] The above-described scenario-oriented mobile application model fusion method is characterized in that step 3) specifically includes the following steps:

[0023] Step 31) Merge the state sets of the execution path model;

[0024] Step 32) Merge the directed jump graphs of the execution path model;

[0025] Step 33) Label the generalized scene path set obtained in step 2) onto the merged model to obtain a larger model that contains more functional scene information.

[0026] By adopting the above technical solutions, the following beneficial effects can be achieved:

[0027] This invention combines the advantages of manual exploration models, which possess execution logic and domain knowledge, with automated exploration models, which have a larger scale and lower cost. It achieves a more accurate and information-rich mobile application execution model at a lower cost, supporting downstream tasks for developers and testers. Attached Figure Description

[0028] Figure 1 This is an example diagram illustrating the construction of an execution path model according to an embodiment of the present invention.

[0029] Figure 2 This is an example diagram of obtaining the traversal starting node in an embodiment of the present invention.

[0030] Figure 3 This is an example diagram of the traversal execution path model in an embodiment of the present invention.

[0031] Figure 4 This is an example diagram of model merging in an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] This embodiment provides a scenario-oriented mobile application model fusion method, which includes the following steps:

[0034] Step 1) Input the application execution data obtained through manual exploration and construct it into an application execution path model, such as... Figure 1 As shown, Figure 1 An example diagram illustrating the construction of an execution path model for an embodiment of the present invention specifically includes the following steps:

[0035] Step 11) Input application execution information explored through manual and automated methods, mainly including three parts: application status information, application jump information, and application metadata. The status is a tuple containing screenshot information and layout structure information at a certain moment during application execution, and each status is identified by a unique number; the jump is a triple, including the jump start state, the jump target state, and the event that triggered the jump. The event that triggered the jump is also a triple, including the interactive controls, the interactive actions, and possible inputs; the metadata includes the application version, name, etc.

[0036] Step 12) Input the application's functional scenario information, which is a sequence of jumps, representing a specific function related to the application;

[0037] Step 13) Construct the execution path model for manual exploration and the execution path model for automated exploration based on the input information in Step 11). Number the states starting from 0 in ascending order, and modify the corresponding numbers in the corresponding jumps. Use the states as nodes and the jumps as edges to construct a directed graph. The directed graph, the screenshot and layout information corresponding to each state, and the application's metadata together constitute the application's execution path model.

[0038] Step 14) Construct functional scenario information based on the information in Steps 11) to 13). The functional scenario information is a sequence of jumps in the application execution path. Each adjacent jump in the sequence is connected end to end to form a path representing a specific function of the application.

[0039] Step 2) Use a breadth-first search algorithm to traverse the application execution path model to obtain the generalized functional scenario path information, which includes the following steps:

[0040] Step 21) On the execution path model of automated exploration, use the interface similarity algorithm to find several interfaces similar to the starting point of the input scene path as starting points, and obtain an example graph of the traversal of the starting node, as shown below. Figure 2 As shown;

[0041] Step 22) Starting from the starting point, perform a breadth-first search. The search depth is limited to the length of the input scene path. In each round of exploration, based on the current exploration depth, select the corresponding state and jump in the scene path. Based on the state similarity algorithm and the jump similarity algorithm, select the node with a similarity exceeding the preset threshold as the successor node of the current exploration node.

[0042] Step 23) After exploration, a tree is obtained. Depth-first search is performed on this tree. Each time a leaf node is reached, the exploration path is recorded as a result path, ultimately resulting in a set of result paths. The result paths from each starting point collectively constitute the generalized functional scenario path. An example diagram of the traversal execution path model is shown below. Figure 3 As shown.

[0043] Step 3) Merge the execution path model from manual exploration with the execution path model from automated exploration, and annotate the generalized functional scenario paths onto the merged model, such as... Figure 4 As shown, Figure 4 This is an example diagram of model merging in an embodiment of the present invention, specifically including the following steps:

[0044] Step 31) Merge the state sets of the two models. To prevent duplicate serial numbers when merging the state sets, different prefixes will be added to the original serial numbers to identify models from different sources.

[0045] Step 32) Merge the jump models and introduce a new pseudo state. The pseudo state does not have a corresponding screenshot file or layout file. It is only used as the entry point of the merged ARP state. When merging the models, add a jump relationship between the pseudo state and the entry state of the original model. Then add the jump relationship of the original model to the new model. The jump graph of the original model becomes a subgraph of the jump graph of the new model.

[0046] Step 33) After merging, based on the set of functional scenario paths obtained in step 2), add the functional scenario labels as attributes of the edges to the directed graph of the application execution path model.

[0047] The above descriptions are embodiments of the present invention, but the specific embodiments described herein are merely illustrative and not intended to limit the invention. Any omissions, modifications, or equivalent substitutions made within the scope of the claims of this invention without departing from the principles and spirit of the invention should be included within the protection scope of this disclosure.

Claims

1. A scenario-oriented mobile application model fusion method, characterized in that, Includes the following steps: Step 1) Before the method is executed, a mobile application modeling specification is predetermined, called the mobile application execution path model. Users need to provide mobile application execution path data obtained through automated and manual exploration, including screenshots and layouts of the interface before and after each execution, as well as the specific operations of each execution, in order to construct the mobile application execution path model. Users also need to provide functional scenario information based on their understanding of the application, that is, an execution sequence that represents a specific functional scenario of the mobile application. Step 1) involves constructing the execution path model, specifically including the following steps: Step 11) Input the application execution information for manual and automated exploration. The application execution information includes the interface information and jump information during execution. The interface information includes the interface screenshot and interface layout. The jump information includes the jump start state, target state and event. Step 12) Input the application's functional scenario information; Step 13) Construct an execution path model for the application based on the information from Step 11). The execution path model includes a directed graph with states as nodes and jump relationships as edges, and the application's metadata. Step 14) Construct functional scenario information based on the information from Steps 11) to 13). The functional scenario information is a sequence of jumps in the application execution path. Each adjacent jump in the sequence is connected end to end to form a path representing a specific function of the application. Step 2) Based on the two application execution path models and scene annotation information obtained in Step 1), a breadth-first search algorithm is used to traverse the automated exploration model. After traversal, several subgraphs matching the input functional scene paths are obtained. Then, depth-first search is used to convert the subgraphs into a set of paths. Step 2) specifically includes the following steps in traversing the execution path model: Step 21) Use an XML layout-based state similarity algorithm, use tree edit distance to measure the similarity between interface layout structures, use character edit distance and histogram distance to estimate the lower bound of tree edit distance, and use the normalized estimated value as the standard for similarity evaluation. Step 22) Use a jump similarity algorithm based on component attribute equivalence and component area overlap rate to determine the degree of jump similarity based on whether the attributes of the components associated with the jump event are equivalent, whether the corresponding areas of the components overlap, and whether the jump event types are the same. Step 23) Use a breadth-first search-based traversal algorithm to find a subgraph in the execution path model that matches the input scene path; Step 24) Use the depth-first search algorithm to convert the subgraph obtained in step 23) into a set of scene paths; Step 3) Merge the two application execution path models obtained in Step 1), and at the same time, label the corresponding functional scenario information onto the merged model according to the path set obtained in Step 2), thus obtaining a merged model with more functional scenario information.

2. The scenario-oriented mobile application model fusion method according to claim 1, characterized in that, Step 3) specifically includes the following steps: Step 31) Merge the state sets of the execution path model; Step 32) Merge the directed jump graphs of the execution path model; Step 33) Label the generalized scene path set obtained in step 2) onto the merged model to obtain a larger model that contains more functional scene information.

Images