Byte code level automatic instrumentation and error log analysis method for mobile platform and application of byte code level automatic instrumentation and error log analysis method

Abstract

The invention discloses a byte code level automatic instrumentation and error log analysis method and application for a mobile platform, and aims to meet the requirements of testers and developers. Automatic instrumentation of different levels, different categories and different granularities is carried out on Android source codes on a real mobile device or a virtual mobile device through a bytecode technology. Application crashes and page logic errors are detected by using automated testing tools or manual testing, and instrumentation logs are collected upon occurrence of errors. Subsequently, re-attempting to reproduce the error and collecting the reproduced instrumentation log. And performing evaluation by using the instrumentation logs of the two times of operation, generating a visual report and analyzing reasons. The method can be used for analyzing the situation that application errors are difficult to reproduce, errors can be positioned and repaired easily, and developers are helped to improve the application quality. In addition, the method can also be used for evaluating the effectiveness of the Android automatic graphic test tool, and is beneficial to improving the tool defects and improving the robustness and reliability of the test tool.

Description

Technical Field

[0001] The present invention belongs to the field of mobile platform automated testing, and in particular relates to a bytecode-level automated instrumentation and error log analysis method and application for mobile platforms. Background Art

[0002] Mobile application testing is a key technology to ensure the quality of mobile applications. A qualified mobile application must undergo systematic and detailed testing to ensure that the system quality meets the expected requirements. Otherwise, once a system containing multiple unrevealed defects is released, it will lead to poor user experience and a large number of user churn.

[0003] Error reproduction is the core link in mobile application testing and is the necessary premise and foundation for correctly locating and fixing errors. In mobile application testing, only errors that can be reproduced can be correctly modified, so that the quality of the system can be improved. However, due to the involvement of diverse, complex and heterogeneous information in runtime or environmental contexts such as the operating system, hardware platform, and the system itself, error reproduction has always been a very challenging research task.

[0004] Existing mobile application testing methods are mainly divided into two categories: manual testing and automated testing. Manual testing requires testers to manually perform all activities step by step and observe whether each step is successfully completed and whether there are any problems in the process. However, the interval time between steps in manual testing is not constant and is greatly affected by the tester, which makes some application errors that rely on precise time control difficult to find and reproduce. Therefore, relying on mobile platform automated testing technology has become a new goal of software research.

[0005] In the field of mobile automated testing, there are currently some research results, which can be divided into the following categories: (1) Automated UI testing tools. Representative studies are Monkey, FastBot, and DriodBot. The Monkey tool can quickly generate pseudo-random user event (such as clicks, taps, or gestures) streams and system-level events to test applications. Fastbot is a model-based testing tool used to model GUI transitions to discover application stability issues. It combines machine learning and reinforcement learning techniques to assist in discovering application errors in a more intelligent way. The DriodBot tool can generate UI-guided test inputs based on dynamically generated state transition models and is a lightweight test input generation tool. (2) Recording and replay tools in Android. Representative studies are VALERA, RERAN, and SARA. VALERA adopts a new technology called sensor-oriented replay, which records and replays sensor and network inputs, event schedules, and inter-application communications through intent to achieve high accuracy and low overhead. RERAN solves this problem by enabling record and replay for the Android smartphone platform. Discovering the gap between industry needs and the performance of publicly available record and replay tools, SARA uses a self-replay mechanism to record more user input information so that it can be accurately replayed without affecting the user experience. While the above tools have been successful in detecting errors and recording replays, the main challenge when fixing these errors lies in stably reproducing these errors, which is critical for developers and testers to fix these errors. Most existing automated UI testing tools lack replay capabilities, requiring testers to manually reproduce any reported crash errors, which can be difficult and labor-intensive. Even for those automated UI testing tools that provide replay capabilities (e.g., Monkey, Ape, DroidBot), they cannot guarantee successful replay due to different runtime or environment contexts. In addition, faced with a large amount of run and replay information, there is no effective automatic comparison mechanism to analyze errors and replay information. The error reproduction problem poses a challenge for testers to diagnose and fix these crash errors. Summary of the invention

[0006] In order to improve and overcome the above-mentioned problems existing in the existing mobile automatic testing technology, the present invention proposes a bytecode-level automatic instrumentation and error log analysis method and application for mobile platforms, which generates corresponding instrumentation codes by traversing different versions of Android SDK and extracting event listeners and event processing functions. During the compilation and construction process, the .class file is intercepted, the target Android application is instrumented at the bytecode level source code, and the instrumented Android application is obtained. Through automated tools or manual testing applications, the test results are recorded and HTML comparison charts are generated to evaluate and reproduce application crashes and page logic errors on the Android mobile platform, assisting testers and developers in locating defects and repairing them.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] The present invention provides a bytecode-level automatic instrumentation and error log analysis method for a mobile platform, comprising the following steps:

[0009] Step 1: Extract Android event listeners Step: Get different versions of Android SDK (Android.jar) (from Android 4.0 to Android 11), traverse all JAR packages, and extract event listeners in the JAR packages;

[0010] Step 2: Extract the event processing functions listened to by all event listeners obtained in step 1, remove duplicates and store them in list L; obtain all life cycle functions and thread execution functions in Android, and form a set A together with the functions in list L, and store them in the SQLite database;

[0011] Step 3: For the event processing functions in set A, classify the components according to the component category to which each function is bound, and write specific plug-in code for each classified component; for the life cycle functions and thread execution functions in set A, also write their specific plug-in code; finally, convert the written code into the corresponding bytecode;

[0012] Step 4: Select an application and determine the location of the instrumentation step: Select an Android application software P and determine the function set T that needs to be instrumented;

[0013] Step 5: Application source code instrumentation step: According to the function set T that needs to be instrumented in step 4, the bytecode obtained in step 3 is inserted into the source code of the Android application software P using the ASM bytecode operation framework to obtain the instrumented Android application software P';

[0014] Step 6: Use automated tools or manual testing to test application crashes or non-crash errors: Use the Android automated graphics testing tool G or manual testing to trigger application crashes or page logic errors E, and record the screen during testing; after the test, record the test cases that produce application crash errors or page logic errors, the instrumentation output file f, the test output results o (including the test output results of automated tools and manual test output results), and the screen recording video v;

[0015] Step 7: Reproduce the error and obtain the instrumentation log step: Reproduce the application crash or page logic error E in step 6, record the instrumentation output file f', test output result o' (including test output result of automation tool and manual test output result), screen recording video v';

[0016] Step 8: Evaluate and analyze the results of the reproduction step: Generate an HTML comparison chart based on the errors that occurred in two or more tests, the stub output file, the test output results, and the screen recording video to evaluate and analyze the reproduction situation.

[0017] Step 1 further includes the following steps:

[0018] Obtain 17 Android SDKs (Android.jar files) from Android 4.0 to Android 11 from the official Android website, use the ASM bytecode manipulation framework to scan all classes in each SDK version, use the regular expression / .+Listener$ / (a string that starts with any character and ends with "Listener") to match all event handling functions, and store them in a SQLite database.

[0019] Step 2 further comprises the following steps:

[0020] Step 2.1, obtain the event processing functions listened to by all event listeners extracted in step 1, record the fully qualified name of each event processing function (the unique identifier of the function, including the package name, class name and method name), the super name of the direct parent class of the class, the version number API Level of the Android SDK, the function signature and other information, and store them in the SQLite database after deduplication, recorded as list L;

[0021] Listing L includes the following event handling functions:

[0022] Context action mode (ActionMode) related functions: public abstract booleanonActionItemClicked(ActionMode mode,MenuItem item);

[0023] Text editing related functions: public abstract void afterTextChanged(Editable s);

[0024] Android system back button related functions: public void onBackPressed();

[0025] Functions associated with check boxes or switch buttons: public abstract void onCheckedChanged(CompoundButton buttonView, boolean isChecked);

[0026] View click event related functions: public abstract void onClick(View v);

[0027] Android navigation component related functions: abstract void onDestinationChanged(@NonNullNavController controller,@NonNull NavDestination destination,Bundlearguments);

[0028] Sliding drawer view related functions: abstract void onDrawerOpened(@NonNull ViewdrawerView);

[0029] Functions related to editing operations of text editor (TextView): public abstract booleanonEditorAction(TextView v, int actionId, KeyEvent event);

[0030] Slide gesture operation related functions: public abstract boolean onFling(MotionEvent e1, MotionEvent e2, float velocityX, float velocityY);

[0031] Function associated with the long press item event of the list view: public abstract void onItemClick(AdapterView<?>parent,View view,int position,long id);

[0032] public abstract boolean onItemLongClick(AdapterView<?>parent,Viewview,int position,long id);

[0033] Function associated with the list view's click item event: protected void onListItemClick(ListView l,View v,int position,long id);

[0034] Functions related to the long press event of the view: public abstract boolean onLongClick(View v);

[0035] Google Map related functions: public abstract void onMapReady(GoogleMapgoogleMap);

[0036] Menu item click event related functions: public abstract boolean onMenuItemClick(MenuItem item);

[0037] Navigation menu item related functions: public abstract boolean onNavigationItemSelected(MenuItem item);

[0038] Options menu item related functions: public boolean onOptionsItemSelected(MenuItemitem);

[0039] Functions related to preference change events: public abstract boolean onPreferenceChange(Preference preference,Object newValue);

[0040] Functions related to preference change events: public abstract boolean onPreferenceClick(Preference preference);

[0041] Runtime permission request related functions: abstract void onRequestPermissionsResult(intrequestCode,@NonNull String[]permissions,@NonNull int[]grantResults);

[0042] Shared preference related functions: public abstract void onSharedPreferenceChanged(SharedPreferences sharedPreferences,String key);

[0043] Web browser view related functions: public void onShowCustomView(View view,WebChromeClient.CustomViewCallback callback);

[0044] Navigation function related functions: public boolean onSupportNavigateUp();

[0045] Functions related to processing long press events: public void verseLongPress(String chapterVerse);

[0046] Step 2.2, obtain all life cycle functions and thread execution functions in Android, form a set A together with the list L, and store it in the SQLite database;

[0047] The life cycle function and thread execution function include:

[0048] Called when the page is created for the first time to complete the initialization of the activity: protected void onCreate(Bundle savedInstanceState);

[0049] Called when the activity changes from invisible to visible: protected void onStart();

[0050] Called when ready to interact with the user: protected void onResume();

[0051] Called when the system is ready to start or resume another activity: protected void onPause();

[0052] Called when the page is completely invisible: protected void onStop();

[0053] Called before the page is destroyed: protected void onDestroy();

[0054] Called when the activity changes from the stopped state to the running state: protected void onRestart();

[0055] A method in the AsyncTask class that is used to perform time-consuming operations in a background thread: protected String doInBackground(Integer...params);

[0056] When a thread is created and an object that implements the Runnable interface is passed to the thread, the thread calls the run method of the object to execute the thread's task: public void run();

[0057] When a thread is created and an object that implements the Callable interface is passed to the thread, the thread calls the object's call method to perform the thread's task and allow it to return a result: public void call().

[0058] In step 3, the steps of classifying and writing instrumentation code include:

[0059] Step 3.1, event handling functions are used to handle user interface events and system events. These functions allow developers to respond to user input and system events in Android applications to perform specific operations or logic. For the event handling functions in set A, classify the components according to the components to which each event handling function is bound. For different components, write specific code. The purpose of the code is to print the current function call time and information that uniquely identifies the component, such as the fully qualified name of the event handling function and the coordinates, class, resource identifier, etc. of the corresponding component, and output this information to the instrumentation output file f. The classified components include View, EditText, AdapterView, RecyclerView, DialogInterface, materialdialogs, TextView, SharedPreferences, among which:

[0060] 1) View: The View event handler is used to handle events related to the View control in the user interface, such as click, touch, slide, etc. The present invention can automatically obtain the parameter variable view of the View, and use String.valueOf(view.getId()) and view.getContext().getClass().getName() to obtain the unique identification information of the View;

[0061] 2) EditText: The EditText event handler is specifically used to handle events in the text input box (EditText), such as text changes, focus gain and loss, etc. The present invention can automatically obtain the parameter variable arg0 of the View, and output the content of arg0 in the instrumentation output, which represents the input content of the application;

[0062] 3) For AdapterView: The AdapterView event handler is used to handle events related to a simple list control, which is a flexible and customizable list control; the present invention can automatically obtain the parameter variables parent and position of the AdapterView, and use parent.getItemAtPosition(position).toString() to print the onItemClick of the AdapterView to obtain the unique identification information;

[0063] 4) RecyclerView: The RecyclerView event handler is used to handle events related to complex list controls. It is a flexible list and grid layout control. The present invention can automatically obtain the parameter variable view of RecyclerView, and use String.valueOf(view.getTag()) and String.valueOf(view.getId()) to obtain the unique identification information of View;

[0064] 5) DialogInterface: The DialogInterface event handler is used to handle dialog box (Dialog) events, such as click events of the OK and Cancel buttons; the present invention can automatically obtain the parameter variables dialog and which of DialogInterface, and output the unique identification information of DialogInterface through dialog.getClass().getName(), String.valueOf(which);

[0065] 6) materialdialogs: materialdialogs is a control for creating a custom dialog box, which provides the option of customizing the dialog box; the present invention can automatically obtain the parameter variables dialog and which of materialdialogs, and output the unique identification information of materialdialogs through dialog.getClass().getName(), String.valueOf(which);

[0066] 7) TextView: The TextView event handler is used to handle events related to the text view (TextView), such as text selection, text click, etc.; the present invention can automatically obtain the parameter variable text of the TextView, and output the unique identification information of the TextView through text.getText();

[0067] 8) SharedPreferences: The SharedPreferences control is used to read and write the shared preferences (SharedPreferences) of an application; the control can be used to store and retrieve application configuration information and user preferences. The present invention can automatically obtain the two parameter variables preference and newValue of SharedPreferences, and obtain the unique identification information of SharedPreferences through preference.getTitle() and newValue.toString();

[0068] Step 3.2: For the lifecycle functions in set A, write specific code to print the calling time and fully qualified name of the current lifecycle function, which indicates that the lifecycle of the application has changed at the current timestamp.

[0069] Step 3.3: For the thread execution functions in set A, write specific code to print the calling time, fully qualified name, thread ID, and return time of the current function, which indicates that a thread is executed during this period of time.

[0070] Step 4 further includes the following steps:

[0071] Step 4.1, determine whether to instrument third-party dependencies: Android developers sometimes reference components provided by third-party libraries. Some default methods of these third-party libraries are directly integrated into the dependent JAR package. Before instrumentation, you can determine whether to instrument the event processing functions provided by these third-party libraries based on specific needs;

[0072] Step 4.2, determine the type of instrumentation: the application code contains event processing functions, life cycle functions and thread execution functions. Before instrumentation, you can determine which categories or specific functions need to be instrumented according to specific needs;

[0073] Step 4.3, determine the granularity of the stub: The event processing function may call some other functions to assist in the response. Before stub insertion, you can determine whether other functions in the event processing function need to be stubbed and printed according to specific needs, and finally obtain a more fine-grained function call chain;

[0074] Step 4.4: According to the requirements in steps 4.1-4.3, determine the function set T that ultimately needs to be instrumented.

[0075] Step 5 further includes the following steps:

[0076] Step 5.1, apply the custom Gradle plug-in to the Android source code, and intercept the .class files and dependent JAR packages compiled by javac during the Transform process;

[0077] Step 5.2: Traverse all intercepted .class files and JAR packages to obtain the function list of each class;

[0078] Step 5.3, traverse the function list of each class, if the current function m is in set A, get the parameter list of m; according to the type of function m, insert bytecode at the entry of function m, print information including the call time and fully qualified name of function m, and output it to the .class file;

[0079] Step 5.4: Pass the modified .class file to the next process of Transform.

[0080] In a specific implementation, the steps of step 5 specifically include:

[0081] Step 5.1: Apply the custom Gradle plug-in to the Android source code, and intercept the .class files and dependent JAR packages compiled by javac during the Transform process. The Android Transform API is an interface reserved by Android for applications before converting .class files into .dex files. The present invention obtains and modifies class files in the interface in the form of a plug-in.

[0082] Step 5.2: Traverse all intercepted .class files and JAR packages to obtain the function list of each class.

[0083] Step 5.3, traverse the function list of each class, if the current function m is in set A, get the parameter list of m. If m is an event processing function, determine the component type bound to m, determine the stub code, rewrite the visitCode() method provided by the ASM framework, insert the corresponding bytecode at the entrance of m, print the call time, fully qualified name and component UI information of m, and output it to the file; if m is a life cycle function, insert the corresponding bytecode at the entrance of m, print the call time and fully qualified name of m, and output it to the file; if m is a thread execution function, rewrite the visitCode() and visitInsn() methods provided by the ASM framework, insert the corresponding bytecode at the entrance and exit of m, print the call time, fully qualified name, thread id and return time of m. For example, onClick(View v) is a typical event processing function, and the UI component type in its parameter list is android.view.View. If onClick is called in a class, the present invention will insert code into the .class file of the corresponding class, print the calling time of the onClick function, the fully qualified name, and the id, className and coordinates of the View component.

[0084] Step 5.4: Pass the modified .class file to the next process of Transform.

[0085] Step 6 Use automated tools or manual testing to check application crashes or page non-crash errors:

[0086] When using an automated testing tool, stop the test after finding an application crash error or page logic error E using the automated tool, and export the output result data, including the test case that generated the application crash error or page logic error, the instrumentation output file f, the test output result o, and the screen recording video v;

[0087] During manual testing, the application is tested manually or with mechanical equipment that assists manual testing. The test is stopped after an application crash error or page logic error E is found. The test case that generates the application crash error or page logic error, the instrumentation output file f, the test output result o, and the screen recording video v are recorded.

[0088] Specifically, the specific requirements for testing with Android automated testing tools are as follows:

[0089] G1: The number of test rounds is 5, and the test duration of each round is 3 hours. The test is stopped after the application crash error or page logic error E is found. Such test round number and duration settings can reduce the impact of the test randomness of the Android automated graphics testing tool, and also enable the tool to have a deeper exploration of the application;

[0090] G2: Export output result data: instrumentation output file f, output result of automated testing tool o, screen recording video v. Instrumentation output file f records specific information of all functions called in the instrumented function set T during the test of the instrumented Android application P', including but not limited to the function call time, fully qualified name, component UI information (such as resourceId, className), etc. The output result o of the automated testing tool includes the test case, crash stack exception information, error exception information, and ANR exception information of this test. The screen recording video v records the entire process of test execution.

[0091] The specific requirements for testing by manual testing method are as follows

[0092] H1: Use manual or mechanical equipment to test the application. The number of test rounds is 5, and the test duration of each round is 1 hour. Stop the test after finding the application crash error or page logic error E.

[0093] H2: Record the test cases that cause application crash errors or page logic errors, the instrumentation output file f, the application output result o (application crash stack exception information, error exception information, etc.), and the screen recording video v during the test.

[0094] An application crash generally refers to a serious error that occurs during the running of an application, causing the application to be unable to continue to run normally. This error usually manifests as the application suddenly closing, exiting, or exhibiting other unexpected behaviors. An application crash may be due to problems with the application itself, such as code errors, memory leaks, out-of-bounds access, etc., or it may be due to device or system problems, such as insufficient memory, incompatible system versions, etc.

[0095] Page logic errors generally refer to non-fatal errors that occur during the operation of an application, which usually do not cause the application to crash or close. This type of error usually manifests itself as certain functions on the page not working properly, abnormal interface display, data errors, etc. Page non-crash errors may be caused by code errors, logic errors, interface problems, etc. in the application, or may be caused by device or network problems.

[0096] In step 7, error reproduction and instrumentation log acquisition further include the following steps:

[0097] When using an automated testing tool to reproduce error E, use the test case replay function provided by the automated testing tool itself, that is, use the random seed value recorded in the test output result o to replay the test case that caused error E, and record the instrumentation output file f', test output result o', and screen recording video v' when reproducing error E for subsequent analysis;

[0098] When the manual test reproduces the error E, the event sequence in step 6 is reproduced manually or with mechanical equipment that assists manual work, and the instrumentation output file f', test output result o', and screen recording video v' when the error E is reproduced are recorded for subsequent analysis.

[0099] The specific steps of step 8 are:

[0100] Step 8.1: Compare o and o'. If the errors generated by the two tests are exactly the same and triggered by exactly the same test case, then it is determined that error E is successfully reproduced in this test. Otherwise, it is determined that error E is not successfully reproduced in this test.

[0101] Step 8.2: If this test fails to reproduce error E, the instrumentation output files f and f' are used to abstract the event execution sequences t and t' respectively. The specific abstraction method is to abstract the millisecond call time of the function and the specific function identification information (i.e., various types of function identification information, such as component UI information, thread ID, etc., which vary depending on the function type) into a set of events, and finally form the event execution sequences t and t'.

[0102] Step 8.3: Compare t and t', align them by events, and visualize them as a sequence of HTML comparison graphs. Find the first event where t and t' differ, find the corresponding position in the screen recording videos v and v' according to the call time in the comparison graph sequence, and troubleshoot the cause of the failure to reproduce. Finally, present the analysis results to the testers and developers of the Android automated graphics testing tool, which will help developers to more directly analyze the reproduction effect and the reasons for the low reproduction rate, thereby helping developers fix the errors.

[0103] The present invention also proposes the application of the above-mentioned automatic plugging and log analysis method in application software error analysis, mobile application automatic testing, etc.

[0104] The beneficial effects of the present invention are:

[0105] The present invention can automatically perform code stubs on functions that need to be recorded in an application, so that when the application calls a function, the calling time and unique identification information of the function can be output, and the output information and screen recording of the test tool can be used to evaluate and analyze application crash errors and page logic errors in the application.

[0106] The present invention will not affect the normal use of the application and will not affect the running speed of the application.

[0107] The present invention provides for the first time a method for analyzing and evaluating the reproducibility problem in mobile automated testing, effectively improving the test efficiency and the adequacy of the test, and can be widely used in the field of mobile application automated testing.

[0108] Due to the lack of detailed function recording tools, the existing technology cannot record the functions used in the process of error occurrence, so it is impossible to capture which functions caused the error, resulting in the inability to reproduce the errors in the application. The present invention uses bytecode automatic stub technology to automatically intercept .class files when the application is built, insert code to print the function execution time and unique identification information, so that when the function is called during testing, the execution time and unique identification information of the function will be output, so as to solve the problem of difficulty in reproduction in mobile automation detection. It is the first technology for fully automatic and effective analysis and evaluation of mobile application error reproduction. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0110] Figure 1 It is a schematic diagram of the bytecode-level automatic instrumentation and error log analysis method for mobile platforms provided by the present invention.

[0111] Figure 2 It is the plugging flow chart in the present invention.

[0112] Figure 3 It is a diagram of an embodiment of the instrumentation output in the present invention.

[0113] Figure 4 It is a diagram of application errors detected by the test tool in the present invention.

[0114] Figure 5 It is a diagram of an HTML comparative embodiment of the present invention.

[0115] Figure 6 It is a diagram of a screen recording comparison embodiment of the present invention. DETAILED DESCRIPTION

[0116] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0117] The present invention discloses a bytecode-level automatic instrumentation and error log analysis method and application for mobile platforms. On a real mobile device or a virtual mobile device, according to the needs of testers and developers, the corresponding Android source code is automatically instrumented at different levels, categories and granularities using bytecode technology. Application crashes and page logic errors are detected by using automated testing tools or manual testing, and instrumentation logs are collected after errors occur. Subsequently, the error is tried again to reproduce and the reproduced instrumentation logs are collected. The instrumentation logs of these two runs are used for evaluation, a visual report is generated and the cause of the error is analyzed. The present invention can be used to analyze situations where it is difficult to reproduce errors in an application, which helps to locate and fix errors and help developers improve application quality. It can also be used to evaluate the effectiveness and reproduction rate of Android automated graphics testing tools, which helps to improve tool defects and improve the robustness and reliability of testing tools.

[0118] Specifically, the bytecode-level automatic instrumentation and error log analysis method for mobile platforms of the present invention comprises the following steps:

[0119] Step 1: Get different versions of Android SDK, traverse all JAR packages in the SDK, and extract event listeners in the JAR packages;

[0120] Step 2: extract the event processing functions listened to by all event listeners obtained in step 1, remove duplicates and store them in list L; obtain all life cycle functions and thread execution functions in Android, and form a set A together with the functions in list L, and store them in the SQLite database;

[0121] Step 3: For the event processing functions in set A, classify the components according to the component category to which each function is bound, and write specific plug-in codes for each type of component, life cycle function, and thread execution function in set A after classification, and convert the written plug-in codes into corresponding bytecodes;

[0122] Step 4: Select an Android application software P and determine the function set T that needs to be instrumented;

[0123] Step 5: According to the function set T that needs to be plugged in step 4, the bytecode obtained in step 3 is inserted into the source code of the Android application software P using the ASM bytecode operation framework to obtain the plugged Android application software P';

[0124] Step 6. Use the Android automated graphics testing tool G or manual testing to trigger an application crash or a page logic error E within the page, and record the screen during the test; after the test, record the test case that caused the application crash error or page logic error, the instrumentation output file f, the test output result o, and the screen recording video v;

[0125] Step 7, reproduce the application crash or page logic error E in step 6, record the instrumentation output file f', test the output result o', and the screen recording video v';

[0126] Step 8: Generate an HTML comparison chart based on the errors that occurred in two or more tests, the instrumentation output file, the test output results, and the screen recording video for evaluation and analysis of the recurrence.

[0127] Example 1

[0128] This embodiment takes the open source software AnkiDroid on GitHub as an example of the detection object (AnkiDroid is a mobile application and a memory card software for efficient learning), and specifically describes the bytecode-based automatic plugging and log analysis method of the present invention. The general whole process is as follows: Figure 1 As shown:

[0129] Step 1: Extract Android event listener steps: Get 17 versions of Android SDK (Android.jar) (from Android 4.0 to Android 11), use the ASM bytecode manipulation framework to scan all JAR packages in each SDK version, use the regular expression / .+Listener$ / (a string starting with any character and ending with "Listener") to match all event handling functions and store them in the SQLite database.

[0130] Step 2: Extract event processing functions, obtain the event processing functions listened to by all event listeners extracted in step 1, record the fully qualified name of each event processing function (unique identifier of the function, including package name, class name and method name), super name of the direct parent class of the class, version number API Level of Android SDK, function signature and other information, and store them in the SQLite database after deduplication, recorded as list L; obtain all life cycle functions and thread execution functions in Android, and form set A together with list L, and store them in the SQLite database;

[0131] Step 3: For the event processing functions in set A, classify them according to the component category to which each function is bound. For each classified component, write specific plug-in code; for the life cycle functions and thread execution functions in set A, also write their specific plug-in code; finally, convert the written code into the corresponding bytecode;

[0132] For the event handling functions in set A, the components are classified according to the components to which each event handling function is bound. For different components, corresponding bytecodes are written. The purpose of the code is to print the current function call time and information that uniquely identifies the component, such as the fully qualified name of the event handling function and the coordinates, class, resource identifier, etc. of the corresponding component, and output this information to the stub output file f.

[0133] For the lifecycle functions in set A, write specific code to print the calling time and fully qualified name of the current lifecycle function, which indicates that the lifecycle of the application has changed at the current timestamp;

[0134] For the thread execution functions in set A, write specific code to print the calling time, fully qualified name, thread id and return time of the current function, which indicates that a thread is executed during this period of time.

[0135] Step 4: Select application and determine the location of the instrumentation step: Given an Android application software P, determine the function set T that needs to be instrumented;

[0136] In this embodiment, the set to be plugged includes the following types of functions, and these functions are constructed into a HashSet set as follows:

[0137] Common method methodSet collection

[0138]

[0139]

[0140]

[0141] Step 5: Application source code instrumentation step: According to the function set T that needs to be instrumented in step 4, such as Figure 2As shown in the instrumentation flow chart, first obtain the AnkiDroid source code, then use the ASM bytecode operation framework to traverse the function list of each class. If the current function m is in set A, obtain the parameter list of m. If m is an event processing function, determine the component type bound to m, determine the instrumentation code, rewrite the visitCode() method provided by the ASM framework, insert the corresponding bytecode at the entrance of m, print the call time, fully qualified name and component UI information of m, and output it to a file; if m is a life cycle function, insert the corresponding bytecode at the entrance of m, print the call time and fully qualified name of m, and output it to a file; if m is a thread execution function, rewrite the visitCode() and visitInsn() methods provided by the ASM framework, insert the corresponding bytecode at the entrance and exit of m, and print the call time, fully qualified name, thread id and return time of m.

[0142] Step 6: Use automated tools or manual testing to test application crashes or non-crash errors: Use the Android automated graphics testing tool G or manual testing to trigger application crashes or page logic errors E, and record the screen during testing; after the test, record the test cases that produce application crash errors or page logic errors, the instrumentation output file f, the test output results o, and the recorded screen video v.

[0143] Use the automated testing tool Monkey for testing. The number of test rounds is 100. The number of test events in each round is set to 10,000, and the event interval is set to 500ms. Stop the test after finding the application crash error.

[0144] After the test is completed, the instrumentation output file f, the output results of the automated testing tool o, and the screen recording video v are recorded. Figure 3 is a diagram of the instrumentation output file f. Lines 1-3 and 4-6 demonstrate the log output when onClick() and onOptionsItemSelected() are called. In this example, line 4 indicates the call time of onOptionsItemSelected(). Line 5 indicates the information of its corresponding UI widget, where the resource id of the Option Menu is "2131296485" and the corresponding Menu Option is marked as "History". Line 6 shows the fully qualified name of the event handler;

[0145] Figure 3 The following figure shows the application error detected by the Monkey automated testing tool, which is derived from the recorded screen video v. The test output result o shows that the error is caused by a null pointer exception, that is, the pointer of the clicked control is null.

[0146] Step 7: Reproduce the error and obtain the instrumentation log step: When using the automated testing tool Monkey to reproduce error E, use the test case replay function provided by the automated testing tool itself, that is, use the random seed value recorded in the test output result o to replay the test case that produced error E. After the test replay, it was found that error E could not be reproduced. At this time, record the instrumentation output file f', test output result o', and screen recording video v' when reproducing error E for subsequent analysis;

[0147] Step 8: Evaluate and analyze the results of the recurrence Step: Generate an HTML comparison chart based on the errors that occurred in the two tests, the output files of the instrumentation tool, the output results of the automated testing tool, and the recorded screen video to evaluate and analyze the recurrence. Figure 5 ) to output. The comparison results in HTML show that in the third event, the two GUI events are different, resulting in different final results of the two runs, that is, the error cannot be reproduced. Figure 5 In the legend, you can check the time when the third event occurred. At this time, in the device screen recording, the time of the event can correspond to the specific time period in the video, and you can determine what caused the event to be different. Figure 6 In the left picture, the rendering state has been completed and the button has been successfully clicked. In the right picture, the button has not been actually clicked because it is in the rendering state, resulting in different results when running the same sequence of events. The reason for this is that the time interval between the two events during the test is set too small, resulting in untimely application rendering. When reproducing, the time interval can be adjusted to increase the recurrence rate, and finally the error can be located and resolved.

[0148] The mobile application in the embodiment can be replaced, including but not limited to various types of open source applications, commercial applications, and personal applications.

[0149] In the embodiment, the number and version of Android versions in the step of extracting the Android event listener in step 1 can be replaced with more versions.

[0150] In the embodiment, the set A can be expanded and can be obtained by traversing the new Android SDK.

[0151] In step 6 of the embodiment, in the step of using an automated tool or manually testing the crash or non-crash error of the application, the automated tool includes but is not limited to the test tool Monkey.

[0152] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A bytecode-level automated instrumentation and error log analysis method for mobile platforms, characterized in that: The method comprises the following steps: Step 1: Get different versions of Android SDK, traverse all JAR packages in the SDK, and extract event listeners in the JAR packages; Step 2: extract the event processing functions listened to by all event listeners obtained in step 1, remove duplicates and store them in list L; obtain all life cycle functions and thread execution functions in Android, and form a set A together with the functions in list L, and store them in the SQLite database; Step 3: For the event processing functions in set A, classify the components according to the component category to which each function is bound, and write specific plug-in codes for each type of component, life cycle function, and thread execution function in set A after classification, and convert the written plug-in codes into corresponding bytecodes; Step 4: Select an Android application software P and determine the function set T that needs to be instrumented; Step 5: According to the function set T that needs to be plugged in step 4, the bytecode obtained in step 3 is inserted into the source code of the Android application software P using the ASM bytecode operation framework to obtain the plugged Android application software P'; Step 6. Use the Android automated graphics testing tool G or manual testing to trigger an app crash or page logic error E within the page, and record the screen during the test; After the test is finished, record the test cases that cause application crash errors or page logic errors, the instrumentation output file f, the test output result o, and the screen recording video v; Step 7, reproduce the application crash or page logic error E in step 6, record the instrumentation output file f', test the output result o', and the screen recording video v'; Step 8: Generate an HTML comparison chart based on the errors that occurred in two or more tests, the instrumentation output file, the test output results, and the screen recording video for evaluation and analysis of the recurrence.

2. The automated instrumentation and error log analysis method according to claim 1, characterized in that: Step 1 further includes the following steps: Get different versions of Android SDK, use the ASM bytecode manipulation framework to scan all classes in each SDK version, use the regular expression / .+Listener$ / to match all event processing functions, and store them in the SQLite database.

3. The automated instrumentation and error log analysis method according to claim 1, characterized in that: In step 2, the following steps are further included: Step 2.1, obtain the event processing functions listened to by all event listeners extracted in step 1, record the information of each event processing function including the fully qualified name, the direct parent class of the class, the version number of the Android SDK, and the function signature, and store them in the SQLite database after deduplication, which is recorded as list L; Step 2.2: Get all life cycle functions and thread execution functions in Android, form a set A together with the list L, and store it in the SQLite database.

4. The automated instrumentation and error log analysis method according to claim 1, wherein: In step 3, the steps of classifying and writing the stub code include: for the event processing functions in set A, classifying them according to the components bound to each event processing function; for the components bound to different event processing functions, the life cycle functions in set A, and the thread execution functions, writing specific stub codes respectively, and converting them into bytecodes.

5. The automated instrumentation and error log analysis method according to claim 1, characterized in that: In step 4, the following steps are further included: Step 4.1: Determine whether third-party dependencies need to be instrumented; Step 4.2: Determine the type and granularity of the pile insertion; Step 4.3: According to steps 4.1-4.2, determine the function set T that ultimately needs to be instrumented.

6. The automated instrumentation and error log analysis method according to claim 1, characterized in that: Step 5 further includes the following steps: Step 5.1, apply the custom Gradle plug-in to the Android source code, and intercept the .class files and dependent JAR packages compiled by javac during the Transform process; Step 5.2: Traverse all intercepted .class files and JAR packages to obtain the function list of each class; Step 5.3, traverse the function list of each class, if the current function m is in set A, get the parameter list of m; According to the type of function m, insert bytecode at the entry of function m, print information including the calling time and fully qualified name of function m, and output it to the .class file; Step 5.4: Pass the modified .class file to the next process of Transform.

7. The automated instrumentation and error log analysis method according to claim 1, characterized in that: In step 6, the test tool tests the application crash or page non-crash error step further including: When using an automated testing tool, stop the test after finding an application crash error or page logic error E using the automated tool, and export the output result data, including the test case that generated the application crash error or page logic error, the instrumentation output file f, the test output result o, and the screen recording video v; During manual testing, the application is tested manually or with mechanical equipment that assists manual testing. The test is stopped after an application crash error or page logic error E is found. The test case that generates the application crash error or page logic error, the instrumentation output file f, the test output result o, and the screen recording video v are recorded.

8. The automated instrumentation and error log analysis method according to claim 1, wherein: Step 7 further includes using an automatic testing tool or manual testing to reproduce error E, including the following steps: When using an automated testing tool to reproduce error E, use the test case replay function provided by the automated testing tool itself, and record the instrumentation output file f', test output result o', and screen recording video v' when reproducing error E for subsequent analysis; When the manual test reproduces the error E, the event sequence in step 6 is reproduced manually or with mechanical equipment that assists manual work, and the instrumentation output file f', test output result o', and screen recording video v' when the error E is reproduced are recorded for subsequent analysis.

9. The automated instrumentation and error log analysis method according to claim 1, characterized in that: The step 8 specifically includes the following steps: Step 8.1: Compare o and o'. If the errors generated by the two tests are exactly the same and triggered by exactly the same test case, it is determined that error E is successfully reproduced in this test. Otherwise, it is determined that error E is not successfully reproduced in this test. Step 8.2, if this test fails to reproduce error E, then the instrumentation output files f and f' are used to abstract the event execution sequences t and t' respectively; Step 8.

3. Compare t and t', align them by events, and visualize them as an HTML comparison chart sequence. Find the first event where t and t' differ. Find the corresponding position in the screen recording videos v and v' according to the call time in the comparison chart sequence, and troubleshoot the cause of the failure to reproduce. Present the analysis results to the testers and developers of the Android automated graphics testing tool.

10. Application of the automated instrumentation and error log analysis method according to any one of claims 1 to 9 in application software error analysis and mobile application automated testing.

Images