Testing method and device based on real machine, electronic equipment and storage medium

Abstract

The invention relates to the technical field of testing, and discloses a testing method and device based on a real machine, electronic equipment and a storage medium, and the method comprises the steps: obtaining the environment configuration information of a testing machine for the real machine; building a code generation environment and an execution environment on the real machine based on the environment configuration information; generating a target code of the to-be-tested application program on the real machine based on the code generation environment; performing iterative execution on the target code based on the execution environment, and determining each target test event corresponding to the to-be-tested application program; and analyzing each target test event, and generating a real machine test result corresponding to the to-be-tested application program. By implementing the technical scheme of the invention, the authenticity of a test environment is ensured, the inconsistency of the bytecode file and an online file caused by the missing or error of the bytecode file is avoided, the test problem caused by the inconsistency of a test machine and a real machine dynamic library is avoided, and the accuracy of a test result is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of testing technology, and in particular to a testing method, device, electronic device and storage medium based on a real machine. Background Art

[0002] At present, the main testing methods for real machine applications are single test generation method and fuzz testing method. However, the above testing methods are usually deployed on a test machine (such as a computer), and the test environment is built on the test machine to simulate the application test on the real machine. Due to the differences between the real machine and the test machine in terms of operating environment, the abnormal events generated by the test machine may not appear on the real machine, and its test accuracy is difficult to guarantee. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a testing method, device, electronic device and storage medium based on a real machine to solve the problem that it is difficult to ensure the accuracy of real machine testing.

[0004] In the first aspect, the embodiments of the present disclosure provide a real machine-based testing method, including: obtaining environmental configuration information of the test machine for the real machine; based on the environmental configuration information, building a code generation environment and an execution environment on the real machine; based on the code generation environment, generating target code of the application to be tested on the real machine; iteratively executing the target code based on the execution environment to determine each target test event corresponding to the application to be tested; analyzing each target test event to generate a real machine test result corresponding to the application to be tested.

[0005] The real machine-based testing method provided by the disclosed embodiment builds a code generation environment and a testing environment for testing applications on a real machine in combination with the environmental configuration information of the test machine, thereby ensuring the authenticity of the test environment. Subsequently, the target code is generated in the code generation environment, and the code is executed in the execution environment, thereby avoiding the inconsistency between the bytecode file and the online file caused by the missing or incorrect bytecode file, and avoiding the testing problems caused by the inconsistency between the test machine and the real machine dynamic library. Since the test is performed on a real machine, the target test events obtained by executing the target code are analyzed to obtain the real machine test results corresponding to the application to be tested, which improves the accuracy of the test results, avoids the reporting of unnecessary problems, and facilitates the technical staff to conduct targeted debugging of the application to be tested based on the test results, further improving the repair efficiency.

[0006] In the second aspect, the embodiment of the present disclosure provides a testing device based on a real machine, including: an acquisition module, used to obtain the environment configuration information of the test machine for the real machine; an environment construction module, used to build a code generation environment and an execution environment on the real machine based on the environment configuration information; a code generation module, used to generate the target code of the application to be tested on the real machine based on the code generation environment; a code execution module, used to iteratively execute the target code based on the execution environment, and determine each target test event corresponding to the application to be tested; an analysis module, used to analyze each target test event, and generate a real machine test result corresponding to the application to be tested.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the real machine-based testing method of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the real machine-based testing method of the first aspect or any corresponding implementation manner thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 is a flowchart of a real machine-based testing method according to some embodiments of the present disclosure;

[0011] Figure 2 is a flowchart of another real machine-based testing method according to some embodiments of the present disclosure;

[0012] Figure 3 is a structural block diagram of a test device based on a real machine according to an embodiment of the present disclosure;

[0013] Figure 4 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0015] At present, the testing methods for real machine applications mainly include unit test generation method and fuzz testing method. Among them, the unit test generation method is based on the analysis of the current code snippet to generate corresponding unit test events (cases) for testing to ensure the correctness of the target code. In order to generate high-quality and high-coverage unit test generation results, the unit test generation method integrates all the current advanced program analysis, testing technologies and algorithms, such as static analysis, dynamic analysis, genetic algorithms, symbolic execution, fuzz testing, mutation testing, etc. The fuzz testing method mainly inputs a series of invalid or random data into the program to try to cause program crashes, errors, memory leaks and other anomalies. It can be mainly divided into two categories: black box testing (parameter generation is purely random) and guided testing (coverage is collected after plugging, and the parameters are guided by coverage feedback).

[0016] However, the above test methods are usually deployed on a test machine (such as a computer), by building a test environment on the test machine to simulate the application test on the real machine. Due to the differences in the operating environment between the real machine and the test machine, the abnormal events generated by the test machine may not necessarily appear on the real machine, and the accuracy of the test is difficult to guarantee.

[0017] Based on this, this public technical solution combines the environmental configuration information of the test machine to build a code generation environment and a test environment for testing the application on a real machine, which ensures the authenticity of the test environment, improves the accuracy of the test results, and avoids reporting unnecessary problems.

[0018] According to an embodiment of the present disclosure, a real machine-based testing method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0019] In this embodiment, a testing method based on a real device is provided, which can be used for electronic devices such as mobile phones, tablet computers, etc. Figure 1 is a flow chart of a real machine-based testing method according to an embodiment of the present disclosure, such as Figure 1 As shown, the process includes the following steps:

[0020] Step S101, obtaining the environment configuration information of the test machine for the real machine.

[0021] The test machine is a device used to test each application unit of the real machine, such as a computer, laptop, etc. The environment configuration information is the test environment built by the test machine to test each unit of the real machine, such as conducting guided fuzz testing (Evodroid Fuzz) on the Android real machine application on the test machine. Specifically, the test environment includes the pre-initialization environment of the test machine, the code generation environment of the test machine (i.e., code generation), and the dynamic execution environment of the test machine (i.e., code execution environment).

[0022] Since the test machine already has a complete code generation environment and dynamic execution environment, in order to quickly build an environment for application testing on the real machine, it can be built based on the test environment framework on the test machine. Specifically, the technician can migrate the environment configuration information of the test machine to the real machine, and correspondingly, the real machine can respond to the technician's migration operation on the environment configuration information of the test machine and obtain the environment configuration information corresponding to the migration operation.

[0023] Step S102: Building a code generation environment and an execution environment on a real machine based on the environment configuration information.

[0024] The code generation environment is an environment built on a real machine to support code development, and the execution environment is an environment built on a real machine to support code compilation and execution. Here, the code generation information and code execution information in the environment configuration information on the test machine can be modified to transform the environment configuration information of the test machine into the environment configuration information suitable for the real machine.

[0025] Combine the modified environment configuration information to build the corresponding code generation environment and execution environment on the real machine. For example, based on the EvoFuzz ​​framework on the test machine, the code generation environment and execution environment of the real machine are built.

[0026] Step S103: Generate target code of the application program to be tested on a real machine based on the code generation environment.

[0027] The target code is the test code written by the technician for the application to be tested on the real machine. Specifically, after completing the construction of the code generation environment, the technician can develop the test code according to the actual test requirements. Correspondingly, the real machine can respond to the technician's code development operations in the code generation environment, create a target object in the code generation environment according to the code development operations, and call the code generation logic in the code generation environment to generate the target code for the application to be tested.

[0028] Step S104, iteratively executing the target code based on the execution environment to determine each target test event corresponding to the application to be tested.

[0029] The target test event is used to characterize the test results obtained by iteratively executing the target code. Specifically, each statement in the target code is executed based on reflection, that is, after the target code is generated, the target code is controlled to be iteratively executed in the execution environment based on the reflection execution mechanism, so that the corresponding execution results and the tracking analysis of the execution results can be obtained in each round of iterative execution, and the target test events generated by the target code of the application to be tested in each round of iterative execution can be determined.

[0030] Step S105: Analyze each target test event and generate a real machine test result corresponding to the application to be tested.

[0031] After obtaining each target test event, each target test event generated in each iteration process is tracked and analyzed to determine the real machine test results generated by testing each target test event on the real machine. Through the real machine test results, the abnormal events existing in the application to be tested on the real machine can be determined. Then, the real machine test results and the test results generated on the test machine are combined to realize the verification of the application to be tested, improve the test accuracy of the application to be tested, and facilitate technicians to optimize abnormal events according to the verification results, realize the closed-loop test of the application to be tested, so as to ensure the running stability of subsequent applications on the real machine.

[0032] The real machine-based testing method provided in this embodiment builds a code generation environment and a testing environment for testing applications on a real machine in combination with the environmental configuration information of the test machine, thereby ensuring the authenticity of the test environment. Subsequently, the target code is generated in the code generation environment, and the code is executed in the execution environment, thereby avoiding the inconsistency between the bytecode file and the online file caused by the missing or incorrect bytecode file, and the dynamic library of the real machine can be directly loaded to avoid testing problems caused by the inconsistency between the dynamic library of the test machine and the real machine. Since the test is performed on a real machine, the target test events obtained by executing the target code are analyzed to obtain the real machine test results corresponding to the application to be tested, which improves the accuracy of the test results, avoids the reporting of unnecessary problems, and facilitates the technical staff to conduct targeted debugging of the application to be tested based on the test results, further improving the repair efficiency.

[0033] In this embodiment, a testing method based on a real device is provided, which can be used for electronic devices such as mobile phones, tablet computers, etc. Figure 2 is a flow chart of a real machine-based testing method according to an embodiment of the present disclosure, such as Figure 2 As shown, the process includes the following steps:

[0034] Step S201, obtaining the environment configuration information of the test machine for the real machine. For detailed description, please refer to the relevant description of the above embodiment, which will not be repeated here.

[0035] Step S202: Building a code generation environment and execution environment on a real machine based on the environment configuration information.

[0036] Specifically, the above step S202 may include:

[0037] Step S2021, obtaining the target class loader running on the real machine and the original class loader in the environment configuration information.

[0038] The target class loader is the class loader required in the real machine running environment, and the original class loader is the class loader required in the test machine running environment. Build a test project for the application on the real machine, migrate the test project of the test machine to the real machine, and query the class loader in the test project of the test machine to determine the original class loader corresponding to the test machine. The target class loader required by the real machine can be determined based on the attribute information of the real machine.

[0039] Step S2022: In response to the replacement operation on the original class loader, the original class loader is replaced with the target class loader based on the replacement operation.

[0040] The replacement operation is an operation in which the technician replaces the original class loader with the target class loader. Due to the difference in the operating environment between the test machine and the real machine, the original class loader required in the test machine is not required in the real machine. Here, the original class loader needs to be replaced according to the operating environment of the real machine.

[0041] Taking the EvoFuzz ​​framework in the test machine as an example, it is necessary to remove all InstructClassloader and sandboxClassloader in the TestGenerationContext class of the test machine to ensure that the class loader Classloader uniformly obtains Thread.currentThread().getContextClassLoader() from the current thread. Since the robolectric operating environment is not required in the real machine, the sandboxClassloader fusion logic needs to be removed at this time. Moreover, the real machine's dvm cannot directly execute bytecode files, and cannot realize dynamic stubbing and loading with InstructClassloader. The stubbing logic needs to be inserted into the bytecode during the compilation stage, and then converted into a dex file, and finally put into the apk package before reaching the stubbing logic. Therefore, the real machine does not support dynamic stubbing of InstructClassloader.

[0042] Among them, InstructClassloader is the class loader in evosuite, which is responsible for loading the target class after inserting various targets such as line coverage and mutation nodes. SandboxClassloader is the class loader in robolectric, which is responsible for loading all shadow classes. The test machine realizes the test of simulating the real machine on the test machine through the original class loader classloader.

[0043] Step S2023, class loading is performed based on the target class loader to obtain the target class environment.

[0044] After the replacement of the original class loading is completed, the current target class loader is used for class loading to obtain the target class corresponding to the target class loader, and the constructor information of the target class is obtained from the real machine. The corresponding target class environment is generated in combination with the constructor information of the target class.

[0045] Step S2024, constructing a code generation environment and an execution environment based on the target class environment.

[0046] After the target class environment is constructed, the code generation logic is loaded in the target class environment to realize the construction of the code generation environment. At the same time, the threads in the original single thread pool for executing the code are initialized to realize the construction of the execution environment.

[0047] All target classes will be loaded in binary format based on the bytecode file through the original class loader's built-in functions, and then the bytecode in the class will be traversed, and the constructors of the class and its related classes will be collected, and the dependency graph will be called, inheritance and other information will be collected. However, considering that the acquisition process of related classes is too complicated and has heavy dependencies, the above only loads the target class corresponding to the target class loader, and does not load its related classes. However, related classes may be needed in the initialization process of building a code generation environment and an execution environment. That is, in the target class loading and initialization stage, the process of traversing the class to collect the constructors of related classes, calling dependency graphs, inheritance and other information is deleted. Only when a target class that has not been initialized is encountered during the code generation environment process, the related class information required to initialize the target class will be supplemented. Accordingly, in some optional implementations, the above step S2024 may include:

[0048] Step a1, detecting whether each target class in the target class environment has completed initialization.

[0049] Step a2: When all target classes have completed initialization, the code generation logic corresponding to the environment configuration information is obtained, and a code generation environment is constructed based on the code generation logic and the target class environment.

[0050] Step a3, initializing the code execution thread corresponding to the code generation environment, and generating an execution environment for the code.

[0051] Step a4: when there is an uninitialized target class, in response to an information supplementation operation for the target class, the target class is initialized based on the information supplementation operation.

[0052] In the process of building the code generation environment, each target class in the target class environment is initialized to determine whether each target class can be successfully initialized. If the target class can be successfully initialized, it means that the initialization of the target class does not depend on other related classes. At this time, the code generation logic of the test machine can be obtained from the environment configuration information of the test machine for reuse to meet the creation of the target object and realize the construction of the code generation environment.

[0053] The code in the test machine is dynamically executed based on the reflection mechanism, and the statements in each type of code implement corresponding dynamic execution. That is, after the generation of each statement in the target code is completed, the original single-thread pool execution thread is initialized normally, and each statement can be reflected and executed on the real machine to obtain the corresponding execution results and tracking analysis of the execution results (i.e. trace analysis).

[0054] If, during the process of building the code generation environment, an incompletely initialized target class is detected, the technician is required to supplement the relevant class information of the target class. Accordingly, the real machine can respond to the technician's information supplement operation for the relevant class information, and determine the relevant class information corresponding to the target class based on the information supplement operation, so as to obtain the constructor of the relevant class, and realize the correct initialization of the target class by combining the constructors of the target class and the relevant class.

[0055] In the above implementation, by detecting whether each target class in the target class environment has completed initialization, whether the related class information needs to be supplemented is determined according to the completion of initialization, thereby avoiding the acquisition of unnecessary related class constructors and simplifying the target class loading process.

[0056] Step S203: Generate the target code of the application program to be tested on the real machine based on the code generation environment. For detailed description, please refer to the relevant description of the above embodiment, which will not be repeated here.

[0057] Step S204, iteratively executing the target code based on the execution environment to determine each target test event corresponding to the application to be tested.

[0058] Specifically, the above step S204 may include: iteratively running the target code using random iteration and / or genetic iteration to generate target test events corresponding to each round of iterative processing.

[0059] Random iteration builds each statement in the code based on the code template, and then performs random mutation and crossover testing without a target to achieve iterative running of each statement in the target code and obtain the target test event corresponding to the random iteration.

[0060] Genetic iteration is based on the reset instrumentation logic to instrument various target indicators such as branches, lines, exception coverage, etc. in the target code to achieve iterative operation of each statement in the target code and obtain the target test event corresponding to the genetic iteration.

[0061] In some optional implementations, the above method may further include:

[0062] Step b1, during the iterative processing of the target code, the execution of the target code corresponding to each target class is started in a single-process mode to generate a single-process execution result.

[0063] Step b2: When there is a target class that fails to execute in the execution result, the target code corresponding to the target class that fails to execute is executed in a multi-process mode to generate a multi-process execution result.

[0064] The execution environment of the real machine has a corresponding task execution mode, specifically, the task execution mode includes a single process mode and a multi-process mode. During the dynamic execution of the target code, the target code corresponding to all target classes is started in the single process mode to start the test task, and the execution status of the target code corresponding to each target class is captured to determine the single process execution result corresponding to each target class.

[0065] If it is detected that there are target classes that have failed to execute (i.e., crash signals of C++ and C) in the single-process execution results corresponding to each target class, the target class that has failed to execute will be stored in the verification list of the multi-process mode, and the breakpoint will be restored to continue to start the test tasks of the remaining target classes in the single-process mode until the target codes corresponding to all target classes have been executed in the single-process mode. At this time, all target classes in the verification list of the multi-process mode can be obtained, and the test tasks for the target classes that have failed to execute can be started in the multi-process mode to execute the target codes corresponding to the target classes that have failed to execute, and generate the corresponding multi-process execution results.

[0066] In the above implementation, the target code corresponding to each target class is dynamically executed in combination with the single-process mode and the multi-process mode, thereby avoiding the phenomenon that the overall application test fails due to the failure of the target class to execute, and ensuring the iterative processing of the target code of each target class to the greatest extent.

[0067] In some optional implementations, the above method may further include:

[0068] Step c1, in response to the process configuration operation of the main process and the sub-process, generating an event execution main process and an event execution sub-process based on the process configuration operation.

[0069] Step c2, iteratively executing each target test event through the event execution sub-process.

[0070] Step c3: when the target test event reports an error in the event execution sub-process, the event execution main process is entered based on the binding relationship between the event execution main process and the event execution sub-process, and the event execution sub-process is killed.

[0071] The process configuration operation is a configuration operation in which the technician configures the event execution main process and the event execution sub-process on the real machine, and builds a binding relationship between the event execution main process and the event execution sub-process to realize the association between the event execution main process and the event execution sub-process.

[0072] Specifically, the activity intermediate bridge (EvoMainActivity) used to run the event execution main process can be configured for implicit startup. After the onCreate method corresponding to the real machine's application Application is executed, the specified implicit main process will be started after a delay of 5 seconds to start the test task for the target test event in the onCreate method of EvoMainActivity.

[0073] The activity intermediate bridge (SubMainActivity) used to run the event execution subprocess can be configured to be explicitly started. It is not started by default and is only started when the target test event needs to be verified. There is a binding relationship between SubMainActivity and EvoMainActivity. SubMainActivity can return the execution results it generates to the EvoMainActivity corresponding to the event execution main process.

[0074] The real machine can respond to the process configuration operation of the technician, and build the corresponding event execution main process and event execution sub-process according to the process configuration operation. Then, after starting the single process mode, each target test event is iteratively executed through the event execution sub-process. If the event execution sub-process does not exist, the corresponding SubMainActivity is created to start the event execution sub-process, and it is bound to the event execution main process at the same time, so as to pass the target test event that needs to be executed to the event execution sub-process for execution.

[0075] If the target test event does not report an error during the execution of the event execution subprocess, the target test event is passed to the event execution main process for execution. If the target test event reports an error during the execution of the event execution subprocess, the specific error stack is obtained from the event execution subprocess, and the event execution main process is returned through the binding relationship between the event execution main process and the event execution subprocess, and then the event execution subprocess is killed.

[0076] It should be noted that after the event execution subprocess is killed, the binding logic needs to be re-determined. Since binding failure will definitely lead to result generation failure, multiple backup checks can be set here to determine whether the binding is successful.

[0077] In the above implementation, the event execution main process and the event execution sub-process are configured to execute the target test event in the event execution sub-process. The event execution main process will only be entered when no error is reported during the execution, thereby avoiding errors in the event execution main process that affect the test process, and further ensuring the iterative execution of the target test event corresponding to the target code.

[0078] Step S205: Analyze each target test event to generate a real machine test result corresponding to the application to be tested. For detailed description, please refer to the relevant description of the above embodiment, which will not be repeated here.

[0079] The real machine-based testing method provided in this embodiment replaces the original class loader in the environment configuration information with the target class loader required for the real machine operation, and loads the target class through the target class loader, thereby realizing the construction of the code generation environment and the execution environment, ensuring the real operating environment of the application, and realizing the direct loading of the real machine dynamic library. The logic implementation is simpler and more effective, avoiding the incompatibility and invalidity problems caused by simulating and generating the real machine dynamic library on the test machine, and enhancing the generation capability of the target code. Furthermore, each statement in the target code is dynamically executed in combination with random iteration and / or genetic iteration, which improves the execution efficiency of the target code and helps expand the test scope.

[0080] In some optional implementations, the above method may further include:

[0081] Step d1, in response to the test project configuration operation on the real machine, building a test project on the real machine based on the test project configuration operation, and configuring the dependent environment of the test project.

[0082] Step d2, in response to the test parameter configuration operation for the real machine, constructing an initialization test environment for the real machine based on the test parameter configuration operation.

[0083] Step d3, compile and adapt the application to be tested on the real machine based on the dependent environment and the initialization test environment to generate an application package of the application to be tested.

[0084] The test project configuration operation is an operation in which a technician creates a new test project on a real machine and configures the project dependencies in the newly created test project. Accordingly, the real machine can respond to the technician's test project configuration operation to build a corresponding test project on the real machine according to the test project configuration operation and configure the dependent environment of the test project.

[0085] In a specific example, create a new Android test project evoConfigPlugin on a real device, package the test task execution project EvoFuzzRun in the test project evoConfigPlugin into a test task package evoFuzzRunCore.jar, and add the dependency of the test task package evoFuzzRunCore.jar in the Android test project evodroidConfigPlugin. At the same time, you can define the startup entry genTask(className,TaskType).

[0086] The test parameter configuration operation is an operation in which a technician configures the parameters required for the test task on the real machine. Accordingly, the real machine can respond to the technician's test parameter configuration operation to build a corresponding initialization test environment on the real machine according to the test parameter configuration operation, that is, a pre-initialization environment.

[0087] Specifically, the test parameter configuration operation may include build configuration (build config), asset configuration (assetsconfig), environment configuration (env config) and application parameter configuration (application config).

[0088] Among them, the build configuration is used to convert source code into executable files, library files or other software components. The build configuration file contains compiler options, dependencies, build targets, output paths, etc. Through the build configuration, technicians can flexibly customize the build process of the test project to meet specific needs and constraints. For example, at this time, you can inject SDK dependencies into the build configuration by modifying the build configuration file corresponding to the path of the main program of the test project. For example, modify the build.gradle file of the path of the main module of the project, and add implementation'com.evo.device:evoDevice:1.0.80' to the first line of dependencies.

[0089] Among them, asset configuration is used to characterize the configuration settings in the application to be tested, which may include asset location, format, permissions and other attribute information. Through these configurations, the software system on the real machine can accurately manage, access and use the assets corresponding to the application to be tested. For example, add the main configuration information main_config.json under the asset configuration directory, and configure the task type task_type="checkCase" to check task_type="fuzzRun". Inject: src / main / assets / differ_class.txt in the asset configuration directory file differ to achieve the path change. Among them, differ_class.txt is the file path of this change, which can be separated by ":". Here is a specific example: "java.com.example.cppmodule.ShowHello:java.com.example.cppmodule.TestBitMap:".

[0090] Among them, the environment configuration refers to the setting of environment variables for the application to be tested in the real machine. Environment variables are some dynamic variables defined in the application to be tested. They store information related to the running environment of the application to be tested, such as paths, database connection strings, API keys, etc. Specifically, the environment configuration file contains various configuration settings required in a specific environment, such as a development environment, a test environment, or a production environment. The environment configuration makes it easy to set appropriate variable values ​​according to different environmental requirements to control and configure the behavior of the application, so that the application has flexibility and scalability in different environments. Specifically, each business can set a compatible test environment separately. For example, for the configuration of no_network: delete the manifest.xml file obtained under the path of the main program in the test project.<uses-permissionandroid:name="android.permission.INTERNET" / > Replace with:

[0091] <uses-permissionandroid:name="android.permission.INTERNET"

[0092] tools:node="remove">.

[0093] Among them, application configuration is used to set and configure the behavior, functions and properties of the application. The application configuration file contains many parameters and options, such as database connection information, log level, authentication method, localization settings, etc. By modifying the application configuration, technicians can adjust the behavior and performance of the application to adapt to different needs and environments. Usually, the application configuration file adopts a specific format, such as XML, JSON or properties file. Through application configuration, the application can be customized and adjusted according to different scenarios and requirements without recompiling or modifying the source code. For example, configure the full path of each business in the application to start the main program MainApplication, and inject in the input interface import: import java.com.example.cppmodule.EvoFuzzMgr, at the same time, search for the onCreate function and inject at the end of the function: EvoFuzzMgr.initEvoTask(this).

[0094] The real machine-based testing method provided in this embodiment implements the pre-initialization of the basic test logic by configuring the test project and test parameters on the real machine, ensuring the compilation adaptability of the application packages of each business and avoiding the problems of inconsistent packaging commands and dependency conflicts.

[0095] In some optional embodiments, the above method may further include:

[0096] Step e1, obtaining the serialization exception event generated by the test machine for the application to be tested, and packaging the serialization exception event into the application package.

[0097] Step e2, deserialize the serialized exception events in the application package to obtain exception events adapted to the real machine.

[0098] Step e3, executing the abnormal event in the execution environment, and determining the target abnormal event generated in the real machine test.

[0099] Step e4: determining the test verification result of the application to be tested based on the target abnormal event and the serialized abnormal event.

[0100] The serialized abnormal event is the serialized result of the abnormal event generated by the test machine during the test of the real machine application. Specifically, the test machine can serialize (dump) the abnormal event generated during the test into a file, and the serialized file represents the serialized abnormal event.

[0101] When configuring resources, store the test event (case) file in the assets directory and package it into the application package apk. Then, the real machine can read the test event (case) file, deserialize the serialized exception event, and obtain the exception event corresponding to the real machine. Input the deserialized exception event into the above-built running environment for running, and record the target exception event that will cause problems when running in the real machine.

[0102] Test the application on the real machine and the test machine at the same time, combine the serialized exception events generated by the test machine and the target exception events generated by the real machine, determine the problems in the testing process, implement test verification for the application to be tested, and obtain the corresponding test verification results.

[0103] The real machine-based testing method provided in this embodiment verifies the serialization abnormal events reported by the test machine in combination with the target abnormal events reported by the real machine, thereby improving the accuracy of the results reported by the test machine.

[0104] In this embodiment, a test device based on a real machine is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0105] This embodiment provides a test device based on a real machine, such as Figure 3 As shown, including:

[0106] The acquisition module 401 is used to acquire the environment configuration information of the test machine for the real machine.

[0107] The environment building module 402 is used to build a code generation environment and an execution environment on a real machine based on the environment configuration information.

[0108] The environment building module 403 is used to generate the target code of the application program to be tested on a real machine based on the code generation environment.

[0109] The code execution module 404 is used to iteratively execute the target code based on the execution environment to determine each target test event corresponding to the application program to be tested.

[0110] The analysis module 405 is used to analyze each target test event and generate a real machine test result corresponding to the application to be tested.

[0111] In some optional embodiments, the environment building module 402 may include:

[0112] The loader acquisition unit is used to obtain the target class loader running on the real machine and the original class loader in the environment configuration information.

[0113] The replacement unit is used to replace the original class loader with the target class loader based on the replacement operation in response to the replacement operation on the original class loader.

[0114] The loading unit is used to load classes based on the target class loader to obtain the target class environment.

[0115] The generation unit is used to build a code generation environment and an execution environment based on the target class environment.

[0116] In some optional embodiments, the generating unit may include:

[0117] The detection subunit is used to detect whether each target class in the target class environment has completed initialization.

[0118] The generation logic acquisition subunit is used to obtain the code generation logic corresponding to the environment configuration information when all target classes have completed initialization, and to build a code generation environment based on the code generation logic and the target class environment.

[0119] The initialization subunit is used to initialize the code execution thread corresponding to the code generation environment and generate an execution environment for the code.

[0120] The information supplement unit is used to initialize the target class based on the information supplement operation in response to the information supplement operation for the target class when there is an uninitialized target class.

[0121] In some optional embodiments, the code execution module 404 includes:

[0122] The iteration unit is used to iteratively run the target code using random iteration and / or genetic iteration to generate target test events corresponding to each round of iterative processing.

[0123] In some optional embodiments, the code execution module 404 may further include:

[0124] The single-process execution unit is used to start the execution of the target code corresponding to each target class in a single-process mode during the iterative processing of the target code, and generate a single-process execution result.

[0125] The multi-process execution unit is used to execute the target code corresponding to the target class that failed to execute in accordance with the multi-process mode when there is a target class that failed to execute in the execution result, and generate a multi-process execution result.

[0126] In some optional implementations, the code execution module 404 may further include:

[0127] The process configuration unit is used to respond to the process configuration operations of the main process and the sub-process, and generate an event execution main process and an event execution sub-process based on the process configuration operations.

[0128] The execution unit is used to iteratively execute each target test event through the event execution sub-process.

[0129] The process switching unit is used to enter the event execution main process based on the binding relationship between the event execution main process and the event execution sub-process, and kill the event execution sub-process when the target test event reports an error in the event execution sub-process.

[0130] In some optional embodiments, the above device may further include:

[0131] The project configuration module is used to respond to the test project configuration operation on the real machine, build the test project on the real machine based on the test project configuration operation, and configure the dependent environment of the test project.

[0132] The parameter configuration module is used to respond to the test parameter configuration operation on the real machine and build an initialization test environment of the real machine based on the test parameter configuration operation.

[0133] The compilation module is used to compile and adapt the application to be tested on the real machine based on the dependent environment and the initialization test environment, and generate an application package of the application to be tested.

[0134] In some optional embodiments, the above device may further include:

[0135] The exception event acquisition module is used to obtain the serialized exception events generated by the test machine for the application to be tested, and package the serialized exception events into the application package.

[0136] The processing module is used to deserialize the serialized exception events in the application package to obtain the exception events adapted to the real machine.

[0137] The target abnormal event determination module is used to execute abnormal events in the execution environment and determine the target abnormal events generated in the real machine test.

[0138] The verification result determination module is used to determine the test verification result of the application to be tested based on the target abnormal event and the serialized abnormal event.

[0139] The electrical real machine-based test device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0140] The further functional description of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0141] The real machine-based test device provided in this embodiment builds a code generation environment and a test environment for testing applications on the real machine in combination with the environmental configuration information of the test machine, thereby ensuring the authenticity of the test environment. Subsequently, the target code is generated in the code generation environment, and the code is executed in the execution environment, thereby avoiding the inconsistency between the bytecode file and the online file caused by the missing or incorrect bytecode file, and the dynamic library of the real machine can be directly loaded to avoid test problems caused by the inconsistency between the dynamic library of the test machine and the real machine. Since the test is performed on the real machine, the target test events obtained by executing the target code are analyzed to obtain the real machine test results corresponding to the application to be tested, which improves the accuracy of the test results, avoids the reporting of unnecessary problems, and facilitates the technical personnel to conduct targeted debugging of the application to be tested based on the test results, further improving the repair efficiency.

[0142] The present disclosure also provides an electronic device having the above Figure 3 The test setup based on a real machine is shown.

[0143] See also Figure 4 , Figure 4 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present disclosure, such as Figure 4 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0144] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0145] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0146] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0147] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0148] The electronic device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0149] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0150] The electronic device also includes a communication interface for the electronic device to communicate with other devices or a communication network.

[0151] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0152] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A testing method based on a real machine, characterized in that: include: Obtain the test machine's environment configuration information for the real machine; Based on the environment configuration information, build a code generation environment and an execution environment on a real machine; Based on the code generation environment, generating target code of the application program to be tested on the real machine; Iteratively executing the target code based on the execution environment to determine each target test event corresponding to the application to be tested; Each of the target test events is analyzed to generate a real machine test result corresponding to the application to be tested.

2. The method according to claim 1, characterized in that The step of building a code generation environment and an execution environment on a real machine based on the environment configuration information includes: Obtain the target class loader running on the real machine and the original class loader in the environment configuration information; In response to a replacement operation on the original class loader, replacing the original class loader with the target class loader based on the replacement operation; Perform class loading based on the target class loader to obtain a target class environment; The code generation environment and the execution environment are constructed based on the target class environment.

3. The method according to claim 2, characterized in that The step of constructing the code generation environment and the execution environment based on the target class environment includes: Detecting whether each target class in the target class environment has completed initialization; When all target classes have completed initialization, the code generation logic corresponding to the environment configuration information is obtained, and the code generation environment is constructed based on the code generation logic and the target class environment; The code execution thread corresponding to the code generation environment is initialized to generate an execution environment for the code.

4. The method according to claim 3, characterized in that Also includes: When there is an uninitialized target class, in response to an information supplementation operation for the target class, the target class is initialized based on the information supplementation operation.

5. The method according to claim 1, characterized in that The iterative execution of the target code based on the execution environment to determine each target test event corresponding to the application to be tested includes: The target code is iteratively run using random iteration and / or genetic iteration to generate the target test event corresponding to each round of iterative processing.

6. The method according to claim 5, characterized in that Also includes: During the iterative processing of the target code, the execution of the target code corresponding to each target class is started in a single process mode to generate a single process execution result; When there is a target class that fails to execute in the execution result, the target code corresponding to the target class that fails to execute is executed in a multi-process mode to generate a multi-process execution result.

7. The method according to claim 5, characterized in that Also includes: In response to the process configuration operations of the main process and the sub-process, generating an event execution main process and an event execution sub-process based on the process configuration operations; Iteratively executing each of the target test events through the event execution subprocess; When the target test event reports an error in the event execution sub-process, the event execution main process is entered based on the binding relationship between the event execution main process and the event execution sub-process, and the event execution sub-process is killed.

8. The method according to any one of claims 1 to 7, characterized in that: Also includes: In response to a test project configuration operation on the real machine, building a test project on the real machine based on the test project configuration operation, and configuring a dependent environment of the test project; In response to a test parameter configuration operation for the real machine, constructing an initialization test environment for the real machine based on the test parameter configuration operation; The application program to be tested on the real machine is compiled and adapted based on the dependent environment and the initialization test environment to generate an application program package of the application program to be tested.

9. The method according to claim 8, characterized in that Also includes: Obtaining a serialization exception event generated by the test machine for the application to be tested, and packaging the serialization exception event into the application package; Deserialize the serialized exception event in the application package to obtain an exception event adapted to the real machine; Executing the abnormal event in the execution environment to determine a target abnormal event generated in the real machine test; Based on the target abnormal event and the serialized abnormal event, a test verification result of the application to be tested is determined.

10. A testing device based on a real machine, characterized in that: include: The acquisition module is used to obtain the environment configuration information of the test machine for the real machine; An environment building module, used to build a code generation environment and an execution environment on a real machine based on the environment configuration information; A code generation module, used to generate target code of the application program to be tested on the real machine based on the code generation environment; A code execution module, configured to iteratively execute the target code based on the execution environment, and determine each target test event corresponding to the application to be tested; The analysis module is used to analyze each of the target test events and generate a real machine test result corresponding to the application to be tested.

11. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the real machine-based testing method described in any one of claims 1 to 9 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the real machine-based testing method according to any one of claims 1 to 9.

Images