Automatic testing method, device and equipment for thermal repair and medium

Abstract

The invention discloses an automatic testing method and device for hot repair, equipment and a medium. The automatic testing method comprises the steps that a test case and an execution script are decoupled into a test case file and a test program; then the test case file is analyzed through the test program, and a test set is generated and used for automatic testing when the to-be-tested device is accessed; then connecting with a plurality of to-be-tested devices at the same time, and distributing an independent task thread for automatic testing for each test device; then performing a hot repair automatic test according to each upgrade scene in the test case file, and generating a test result; and generating a test report according to the test result. According to the method, simultaneous development is supported by decoupling the test cases and the execution scripts, the cooperation efficiency is improved, a multi-device access scheme is adopted during automatic testing, different hardware devices are supported to be tested at the same time, the compatibility testing is efficiently executed, and the efficiency is improved by times.

Description

Technical Field

[0001] The present invention relates to the technical field of software upgrade testing, which is applicable to the financial field, and in particular to an automatic testing method, device, equipment and medium for hot repair. Background Art

[0002] Hot fix is ​​a technical method to repair urgent problems in software or systems without interrupting users' normal use of applications (such as a bank's APP) or services. This repair method allows developers to quickly respond to and resolve bugs or security issues that occur online without requiring users to manually download and install the entire application update package.

[0003] In Android development, hot fix technology is particularly important. Android hot fix technology is a software online upgrade technology that silently updates the online version. It is mainly used to deal with some sudden and urgent problems. After the hot fix, the difference between the two versions before and after the hot fix will be compared, and a patch package will be generated. Then, when the application (i.e. APP) is running, the patch package will be dynamically integrated into the application's running environment, and the classes or methods in the patch package will be used to replace the existing classes or methods, so as to achieve the effect of hot fix.

[0004] Android hot fix is ​​an important but low-frequency function. Before hot fix is ​​performed on the user side, the corresponding hot fix needs to be performed on the test terminal first, and after passing the test, the user side can be hot fixed. At present, hot fix testing is completed manually, and the test process is complex, lengthy, cumbersome, and time-consuming. Testers need to operate on multiple platforms (such as Android platform, iOS platform, Web platform, etc.), and at the same time, they need to obtain data information or status information from non-response data to verify the status and progress of the current hot fix. This requires testers to be familiar with the system under test and master relevant testing techniques to complete the test task of the hot fix function. Moreover, the test process is complicated, downloading differential packages takes a long time, and the test cost is high. Because of the high test cost, it is often not taken seriously in the test planning, and only rough use cases are arranged for execution, which lays a hidden danger for whether the subsequent user-side hot fix is ​​successful. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide an automatic testing method, device, equipment and medium for thermal repair that can be applied to financial technology or other related fields, aiming to solve the problem that the existing thermal repair cannot be automatically tested and the testing efficiency is low.

[0006] The technical solution of the present invention is as follows:

[0007] In a first aspect, the present invention provides an automatic testing method for thermal repair, comprising:

[0008] Decouple test cases and execution scripts into test case files and test programs;

[0009] Parsing the test case file through the test program and generating a test set for performing automated testing when the device to be tested is connected;

[0010] Connect to multiple devices to be tested at the same time and assign independent automated test task threads to each test device;

[0011] Performing hot fix automation testing according to each upgrade scenario in the test case file and generating test results;

[0012] A test report is generated according to the test results.

[0013] In a second aspect, the present invention provides an automatic testing device for thermal repair, comprising:

[0014] Decoupling module, used to decouple test cases and execution scripts into test case files and test programs;

[0015] A parsing module, used to parse the test case file through the test program and generate a test set for automatic testing when the device to be tested is connected;

[0016] The communication and control module is used to connect to multiple devices to be tested at the same time and allocate an independent automated test task thread to each test device;

[0017] An automated testing module, used to perform automated hot fix testing according to each upgrade scenario in the test case file and generate test results;

[0018] The report generation module is used to generate a test report based on the test results.

[0019] In a third aspect, the present invention provides an automatic test device for thermal repair, the device comprising at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned automatic testing method for hot repair.

[0022] In a fourth aspect, the present invention provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to perform the above-mentioned automatic testing method for hot repair.

[0023] Beneficial effects: The present invention discloses an automatic testing method, device, equipment and medium for hot repair. Compared with the prior art, the present invention decouples test cases and execution scripts, supports the simultaneous development of test cases and execution scripts, improves collaborative efficiency, and adopts a multi-device access solution during automated testing to support simultaneous testing of different hardware devices, thereby achieving efficient execution of compatibility testing and doubling the efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0025] Figure 1 A flow chart of an automatic testing method for thermal repair provided by an embodiment of the present invention;

[0026] Figure 2 A flowchart of step S200 in the automatic testing method for thermal repair provided by an embodiment of the present invention;

[0027] Figure 3 A flowchart of step S300 in the automatic testing method for thermal repair provided by an embodiment of the present invention;

[0028] Figure 4 A flowchart of step S400 in the automatic testing method for thermal repair provided by an embodiment of the present invention;

[0029] Figure 5 Flow chart of step S402 in the automatic testing method for hot repair provided by an embodiment of the present invention;

[0030] Figure 6 A schematic diagram of the functional modules of the device for automatically generating test data provided by an embodiment of the present invention;

[0031] Figure 7 A schematic diagram of the hardware structure of a device for automatically generating test data provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below in conjunction with the accompanying drawings.

[0033] The automatic testing method for hot repair provided by the present invention can be used for the application system or operating system of smart terminals such as Android system, ISO system (iPhone operating system, iOS system), and the automated testing of the hot repair program installed and applied therein, and can improve the efficiency, accuracy and reliability of hot repair testing.

[0034] The present invention uses yaml language to write test cases, which reduces the difficulty of test case writing, so that testers can complete the writing of test cases without coding, and implements hot repair testing of USB-linked and WiFi-linked hardware devices through Android Debug Bridge (ADB) technology, and allows hardware devices of different operating systems and different manufacturers to be tested simultaneously, thereby improving test efficiency.

[0035] The present invention supports multiple upgrade scenarios, including user ID upgrade, APP ID upgrade, full upgrade, specified version upgrade, etc., and realizes automatic verification of the entire process. During testing, it combines back-end interface status query, real-time Android log query and system information ADB query to accurately judge the hot repair status and progress in an unattended environment.

[0036] When the differential package download fails, the differential package can be re-pulled and a hot fix can be triggered through Broadcast (system broadcast), which improves the completion rate of use case execution, thereby reducing testing costs, improving problem troubleshooting efficiency, and reducing bug reproduction processes.

[0037] In order to solve the above problems and achieve the above effects, the present invention proposes an automatic testing method for hot repair. Figure 1 , Figure 1 The present invention provides a flowchart of an embodiment of the automatic testing method for thermal repair.

[0038] The automatic testing method for hot repair provided in this embodiment is applied to intelligent mobile devices, fixed terminal devices, etc. The terminal operating system for hot repair may include a handheld device operating system, such as an IOS system, an Android system, or other operating systems, including but not limited to smart phones, car computers, tablet computers, portable computers, and desktop servers, etc. Figure 1 As shown, the test method specifically includes the following steps:

[0039] S100, decoupling the test case and the execution script into a test case file and a test program.

[0040] Taking the Android operating system as an example, in an optional embodiment, the test case is used to verify the specific functions or behaviors of the Android application, with the aim of ensuring that the application works as expected when performing a hot fix update and maintains its functionality and performance after the update.

[0041] The contents of the test cases mainly include:

[0042] Test Purpose: Clarify the functionality or fix that the test case is intended to verify.

[0043] Preconditions: Conditions that need to be met before executing a test case, such as the initial state of the application or necessary configuration.

[0044] Test steps: A detailed description of each operation step required to perform the test. In addition to automatic testing, it may also include user interface interactions, API (Application Programming Interface) calls, etc.

[0045] Expected results: Define the expected results after the test execution to verify whether the behavior of the hot-fixed application meets the requirements.

[0046] Actual results: The actual output or behavior recorded during test execution for comparison with expected results.

[0047] Test results: Determine whether the test succeeded or failed based on the comparison between the expected results and the actual results.

[0048] Exception handling: If exceptions or errors occur during testing, the test case should contain steps to handle these situations.

[0049] Log records: record key information during the test, including operation steps, timestamps, system status, etc.

[0050] In step S100, the test case is written in yaml language and decoupled from the execution script (test program), making the test case easy to write and maintain, while allowing non-developers (such as test experts) to participate in the creation of test cases. Through the test program, these yaml format test cases are parsed and converted into the format required for automated test execution, such as json, and then executed by the test program, which improves the flexibility and maintainability of the test case and supports the parallelization of testing and development work.

[0051] In an embodiment of the present invention, the execution script is a program or script file used to automatically execute a test task, that is, a test program, which can parse a test case file (such as a yaml format) and automatically execute specific test operations according to the instructions in the test case.

[0052] The execution script contains commands for controlling connected Android devices (such as mobile phones, car computers, tablets, etc.), such as sending instructions through ADB. It can manage the data used in the automated testing process, including preparing test data, cleaning up the test environment, etc.

[0053] The execution script may also include logic to verify whether the test results meet expectations, such as checking whether the version of the application (such as a bank APP, chat APP, etc.) or the operating system (such as the Android operating system) is updated, checking the key information in the application log, etc. During the test process, the execution script also includes an error detection and handling mechanism to perform appropriate recovery operations or record failure information when the test fails. During the test process, the execution script is also responsible for recording key information during the test, including test steps, status, and any abnormal conditions, for subsequent analysis, and generating a test report after the test is completed, summarizing the test results and related information.

[0054] During the execution of the automated testing method of the present invention, the execution script is a key component for realizing automated testing, thereby reducing manual intervention and improving testing efficiency and accuracy.

[0055] In an optional embodiment, the test case is written in yaml language and is decoupled from the execution script (test program). The choice of this format makes the test case easy to write and maintain, while allowing non-developers (such as test experts) to participate in the creation of test cases. Through the test program, these yaml-formatted test cases are parsed and converted into the format required for automated test execution, such as json, and then executed by the test program. This decoupling method improves the flexibility and maintainability of the test case and supports the parallelization of testing and development work.

[0056] When writing test cases, you need to use the YAML language to write test cases, organize test steps, expected results and other information in an easy-to-read and easy-to-write format, and define clear field names and field meanings, as well as required and optional fields in the YAML test case file.

[0057] In one embodiment, before step S100, the method further includes:

[0058] Store test case files separately from test programs to facilitate parallel development and maintenance;

[0059] Define the name, meaning, required and optional fields of the fields in the test case file.

[0060] The test program must include parsing logic corresponding to the fields defined in the yaml test case file, and define a protocol between the yaml test case file and the test program to clarify the name, meaning and data format of the fields, so that during subsequent automated testing, the test program can read the yaml test case file and parse the test case definitions in the file.

[0061] S200, parsing the test case file through the test program, and generating a test set for performing automated testing when the device to be tested is connected.

[0062] During the specific implementation, the test cases and test programs must follow the agreed protocol, including the agreed field names, field meanings, required and optional fields, etc. The test program generates a json test set by parsing the yaml file. The test program parses the yaml case file and generates a json format test set for test execution according to the protocol. The test cases and test programs are maintained separately to support simultaneous development and improve collaborative efficiency.

[0063] In actual applications, json test sets can be written by testers or automatically generated by some tools. The automated test program will parse the json test set, execute the test steps according to the instructions, verify the expected results, and then write the actual results and test status back to the json file for analysis and reporting, which improves the efficiency and accuracy of the test.

[0064] like Figure 2 As shown, in specific implementation, the step S200 includes:

[0065] S201, parsing the test case file by the test program;

[0066] S202, generating a json test set according to the parsed test case file;

[0067] S203, parsing the JSON test set for automated testing when the device to be tested is connected.

[0068] Optionally, a json test set is a set of test data and related instructions expressed in json (JavaScript Object Notation) format, which is used to guide the automated test program to execute test cases. And json is a lightweight data exchange format that is easy for people to read and write, and also easy for machines to parse and generate.

[0069] Among them, the json test set includes the process identifier of each test case, a description of the purpose and operation of the test case, the conditions that need to be met before executing the test case, and detailed operation steps, including user interface operations, API calls, etc.

[0070] In addition, the json test set also includes the expected output or behavior after the test is executed, the actual output or behavior after the test is executed, whether the test passes, fails, or has other status, the error description and stack trace when the test fails, and the configuration information of the test environment or test data.

[0071] The advantages of JSON test sets include: JSON format has a clear hierarchical structure and is easy to understand and write; JSON is independent of programming languages ​​and can be parsed and generated by multiple languages; JSON test sets are easy to integrate with automated testing frameworks and continuous integration / continuous deployment (CI / CD) systems; and test data and steps can be easily added or fixed as needed.

[0072] S300, connecting to multiple devices to be tested simultaneously, and allocating an independent automated test task thread to each test device.

[0073] The present invention obtains USB-linked and WiFi-linked hardware devices (i.e., devices to be tested) through ADB, allocates an independent task thread to each device of the Android system, such as a mobile phone, a car computer, a tablet, etc., and allows multiple devices to run simultaneously, thereby realizing simultaneous testing of hardware devices of different operating systems and different manufacturers. By performing hot repair tests on hardware devices of different operating systems and different manufacturers, it can be verified whether the system or application to be hot repaired is compatible with various hardware devices, thereby avoiding abnormalities when the user end starts the system or the corresponding application for hot repair. The present invention saves test time and improves efficiency by testing multiple devices in parallel.

[0074] Since the testing method of the present invention supports parallel testing of multiple devices, the execution script also includes codes for managing multiple device testing threads.

[0075] In one embodiment, Figure 3 As shown, step S300 includes:

[0076] S301, connecting multiple devices to be tested simultaneously through a USB interface and / or WiFi, and dynamically assigning a process identifier to each device to be tested;

[0077] S302: Create an independent automated test task thread for each device to be tested according to the process identifier to implement concurrent testing.

[0078] The process ID is a unique identifier. In the Android system, each running application has at least one process associated with it. The running status of each application can be identified by dynamically allocating a PID (i.e., a process ID). In this embodiment, the dynamically allocated PID is composed of some numbers. When generating a process ID, the following aspects should be ensured:

[0079] 1) Every process has a unique PID, even if it is two different applications or services running on the same device.

[0080] 2) The PID remains unchanged during the existence of the application process until the process is terminated by the system or ends running.

[0081] 3) The operating system uses PID to manage resource allocation, scheduling, and communication of processes.

[0082] 4) Developers and system administrators can use PID to monitor and debug application behavior, such as by viewing logs, analyzing memory usage, or sending signals.

[0083] 5) In Android logs (Logcat), the PID is usually displayed together with the log messages to help developers determine which messages come from which process of which application.

[0084] 6) You can query the PID of a specific application through tools such as ADB (Android Debug Bridge) to perform related operations or checks.

[0085] However, by monitoring the changes in PID, it is possible to determine whether the hot-fixed program has been started or terminated. It can also be used to check whether the application runs with a new process ID after the upgrade, thereby verifying whether the upgrade is successful. It can also monitor the running status of the application to ensure that it runs normally without crashing.

[0086] S400: Perform hot fix automation testing according to each upgrade scenario in the test case file, and generate test results.

[0087] Among them, the upgrade scenarios include but are not limited to user ID upgrade, hot fix program ID upgrade, full upgrade, specified version upgrade, abnormal crash (re-decompression of data / hot fix rollback) processing, deletion of download packages, replacement of upgrade packages, etc., covering all business scenarios of hot fixes, and realizing full-process automated verification without manual intervention.

[0088] In one embodiment, if Figure 4 As shown, the step S400 includes:

[0089] S401. Configure test tasks according to various upgrade scenarios.

[0090] In an optional embodiment, the test tasks are configured for different Android hot fix upgrade scenarios, and the configuration method is as follows:

[0091] When upgrading a user ID, configure a test task to simulate the upgrade process of a specific user ID to ensure that the hot fix can be performed for a specific user.

[0092] In the APP ID upgrade scenario, configure the test task to simulate the upgrade based on the application ID to ensure that different applications or different versions of applications can receive the correct hot fix updates.

[0093] For the full upgrade scenario, configure the test task to simulate a full upgrade, that is, all users receive the complete application update package.

[0094] When upgrading a specific version, configure the test task to simulate upgrading only the application with a specific version number to ensure version compatibility and upgrade correctness.

[0095] When dealing with abnormal crash scenarios, configure test tasks to simulate abnormal crashes of applications during upgrades, and test whether the system can correctly roll back to a stable state and re-decompress data or perform hot upgrades (i.e., hot repairs).

[0096] For the download package deletion scenario, configure a test task to simulate the situation where the download package is deleted during the upgrade process to test whether the system can handle this situation and continue the upgrade process.

[0097] When replacing an upgrade package, configure a test task to simulate replacing the upgrade package during the upgrade process to test whether the system can adapt to this change and perform the upgrade correctly.

[0098] To cover all business scenarios of hot fixes, configure test tasks to cover all possible hot fix business scenarios to ensure that automated testing can fully verify the hot fix process.

[0099] When implementing full-process automated verification scenarios, configure test tasks to implement full-process automated verification from triggering the upgrade to completing the upgrade without manual intervention.

[0100] For detailed statistics and test report scenarios, configure test tasks to collect the execution results of each case and output detailed test reports based on these results to help quickly locate problems.

[0101] S402: During hot repair, the hot repair status and progress are verified through backend interface status query, real-time Android log query and system information ADB query.

[0102] After the hot upgrade task is triggered (such as when a bank APP is launched by the user, hot repair is triggered), the test system will poll the background interface within a certain period of time to obtain the push time. If the push time is not obtained within the specified time, the hot upgrade failure is triggered, the subsequent process is terminated, and the failure information is recorded. If the device has received the hot upgrade task, it will enter the upgrade process.

[0103] Taking the Android system as an example, the Android log is the recorded information generated by the system and applications. It is mainly used to record system-level events, such as startup, shutdown, system updates, errors and other system messages, and detailed information of application runtime, including user interface operations, API calls, network requests, etc. It also records application crashes, system errors, uncaught exceptions and other information. Therefore, the Android log can be used to track and diagnose events, behaviors and problems on the device to be tested.

[0104] Android logs can be accessed in a variety of ways, including using the "Logcat" tool on an Android device, through the ADB (Android Debug Bridge) command-line tool, or using the logging API in your application to generate and retrieve logs. Android logs are a key resource in automated testing and problem diagnosis because they can provide real-time data about application behavior and potential problems.

[0105] When querying system information through ADB, you can perform a variety of queries and operations through Android Debug Bridge (ADB) commands, such as:

[0106] Within a certain period of time, poll the specified directory / Cache / download to detect the download status of the hidden file differential package tmp file, and determine whether the differential package is downloaded successfully through the change of file size and md5 value verification.

[0107] Get the log file generated by the APP through ADB instructions, query key log information, and determine whether the APP has performed a differential package upgrade (including differential package decompression, verification, and merging to generate a complete package).

[0108] Through ADB commands, poll the specified directory / install within a certain period of time to detect whether the APK installation package exists, obtain the log files generated by the APP and request the background interface through ADB commands to obtain the current version information, and compare the version information obtained from the two channels to determine whether the upgrade is successful.

[0109] Use ADB commands to detect changes in PID and running memory during APP operation over a period of time to determine whether the APP runs normally after the upgrade.

[0110] In step S402, when querying and verifying the hot repair status and progress, first obtain the push time by polling the backend interface; then, detect the differential package download status through ADB instructions; then analyze the log file and system information to verify the upgrade status.

[0111] Specifically, the step S402 includes:

[0112] S4021. After the hot repair task is triggered, poll the backend interface to obtain the hot repair push time;

[0113] S4022. When the hot fix push time is obtained, the differential package download status is detected through ADB instructions, and whether the differential package download is successful;

[0114] S4023. When the differential package is downloaded successfully, obtain key log information in the log file generated by the hot fix program through an ADB command, and determine whether the hot fix program performs a differential package upgrade or a full package upgrade during hot fix according to the key log information;

[0115] S4024, when the hot fix program is identified to perform a full package upgrade, detecting whether the full upgrade installation package exists through an ADB command;

[0116] S4025. When the full upgrade installation package is detected, the thread identification change and running memory during the operation of the hot fix program are detected through ADB instructions, and whether the upgraded program runs normally is determined based on the thread identification change and running memory.

[0117] During the test, the ADB command is used to detect the PID changes after the application upgrade. If the PID remains unchanged and the running memory reaches the normal operating memory, the upgrade can be considered successful. If the PID changes or the running memory is abnormal, it may mean that the upgrade has failed or there is a problem in the application operation.

[0118] In step S4021, the test system queries the backend interface to obtain the time when the hot fix update package (differential package or full package) is actually pushed to the user device. Therefore, the push time is a key indicator in the hot fix process, indicating that the upgrade package has begun to be distributed to the user device.

[0119] Specifically, in the automated testing method of the present invention, the push time may include:

[0120] The time when the hot fix differential package or full package is generated on the server, the time when the hot fix update package begins to be distributed to the user device through the network, the time when the user device actually receives the update package, which may vary depending on network conditions and other factors, and the push event time recorded by the back-end system, which is used to track and monitor the upgrade process, etc.

[0121] In the automated testing method, obtaining the push time can verify whether the upgrade process is started as expected, determine whether the upgrade package is available for download to the user's device, monitor and record the entire process from the start of the push to the user's device receiving the update package, and determine whether the push is completed within the scheduled time to check the efficiency and reliability of the distribution process.

[0122] In step S4022, the ADB instruction is used to monitor the downloading of the differential package, and the file size change and MD5 value are used to verify whether the download is successful. Specifically, the present invention uses the ADB instruction to poll the specified directory / Cache / download within a certain period of time (such as within 1 minute), detect the downloading of the hidden file differential package tmp file, and determine whether the differential package is successfully downloaded through the tmp file size change and md5 value verification. If the file size no longer changes within a certain period of time, the md5 value of the file is calculated to compare whether the current md5 value is consistent with the expected md5 value. If they are consistent, the download of the differential package is successful, otherwise, the download of the differential package fails. When the differential package is downloaded successfully, it means that the test equipment hot repair adopts the differential package upgrade. When it is determined that the differential package download fails, it is identified that the hot repair is upgraded through the full package.

[0123] In step S4023, when the differential package download fails, step S4023 may further include:

[0124] Delete the downloaded abnormal files;

[0125] Re-pull the differential package from the server;

[0126] Trigger hot fixes via Broadcast.

[0127] When it is known that the differential package or the complete package cannot be downloaded normally due to device network anomalies, step S4023 may further include:

[0128] Access the installation package storage service through a computer to download the package;

[0129] Push the package to the specified directory on the device;

[0130] Trigger silent upgrade through ADB broadcast command.

[0131] In step S4023, when determining whether the hot fix is ​​to be upgraded using a differential package, the device's log file can be first obtained through ADB instructions, and key log information can be extracted; then the log information is analyzed to determine whether the APP has correctly executed the differential package upgrade process.

[0132] In step S4025, when judging whether the hot fix program can run normally, the version information of the APP is obtained and whether the upgrade is successful through log file analysis and ADB instructions, the PID changes during the operation of the APP are detected, and the running status of the APP after the upgrade is evaluated.

[0133] During the hot fix process, differential package upgrade and full package upgrade are two different APP update methods. For example, differential package upgrade can reduce data transmission volume and save user bandwidth, increase update speed, and full package upgrade requires downloading and installing the complete application installation package (APK). This upgrade method does not depend on the application version currently installed by the user.

[0134] A differential package is usually smaller than a full package because it only contains the changed parts, but the differential package needs to be compatible with the user's current application version, while the full package does not need to consider this. Due to its small size, differential packages are usually faster to download and update, but the generation and application of differential packages may be more complicated, and it is necessary to ensure that they are correctly applied to existing applications.

[0135] When the test hot fix is ​​completed, the step S400 further includes:

[0136] S410, obtaining the hot fix program version information in the log file generated by the hot fix program through an ADB command, and obtaining the current version information of the hot fix program through a backend interface, and comparing whether the two version information are the same;

[0137] S420: When the two version information are the same, it is identified that the hot fix program is upgraded successfully; when the two version information are different, it is identified that the hot fix program is upgraded failed, and failure information is recorded.

[0138] The present invention obtains the version information in the log file generated by the APP through the ADB command, and requests the background interface to obtain the current version information, and then compares the version information obtained from the two channels. If they are consistent, it means that the upgrade is successful. If they are inconsistent, the failure information is recorded.

[0139] After the hot fix, the step S400 further includes:

[0140] Use ADB commands to detect the changes in the PID and running memory of the APP during a period of time to determine whether the APP runs normally after the upgrade. If the PID does not change and the running memory reaches the normal running memory, the upgrade is successful. Otherwise, the upgrade fails and the failure information is recorded.

[0141] S500: Generate a test report according to the test results.

[0142] In an optional embodiment, the generation of the test report includes the following steps:

[0143] Refine the statistics of the hot repair process status and divide it into four stages: trigger, download, upgrade, and restore;

[0144] Statistics of the execution results of each test case;

[0145] Output test report based on statistical results.

[0146] Generating a test report is also an important part of the automated testing of the present invention. The detailed statistical report of the test report can help testers quickly locate the problem, thereby accelerating the problem-solving process. The report provides detailed information on the abnormal links and execution steps, which can reduce the need for bug reproduction. In addition, the environmental information and log information contained in the test report help developers understand the context in which the problem occurred, thereby diagnosing the problem more accurately.

[0147] At the same time, the test report is based on the recorded information analysis and outputs a relatively accurate description of the problem, such as network anomalies, failure to synthesize differential packets into complete packets, hot loading anomalies, etc. This helps to quickly identify the cause of the problem, so that the hot fix program can be modified and pushed to the user end for hot fixes after passing the test. In addition, the report can automatically identify download anomalies caused by network problems and provide solutions to reduce manual intervention.

[0148] In order to better understand step S400, the hot fix automated test is described in detail below:

[0149] Step A: After the hot repair task is triggered on the device to be tested, the backend interface is polled as soon as possible to obtain the push time through the backend interface status query.

[0150] If the push time is not obtained within the specified time, the hot repair failure is triggered, the subsequent process is terminated, and the hot repair failure information is recorded 1. On the contrary, the device to be tested has received the hot repair task and entered the hot repair upgrade process.

[0151] Step B: poll the specified directory / Cache / download within the second time through the ADB command to detect the download status of the hidden file differential package tmp file.

[0152] Specifically, the difference package can be downloaded successfully by checking the file size change and the md5 value. If the file size does not change within the third time, the md5 value of the file is calculated to compare whether the current md5 value is consistent with the expected md5 value. If they are consistent, the difference package is downloaded successfully; otherwise, the difference package fails to be downloaded, and the hot fix failure information 2 is recorded. Then proceed to the next step of verification.

[0153] When the download of the differential package fails, delete the downloaded abnormal file, re-pull the corresponding differential package from the server, and trigger the hot fix through Broadcast, so as to avoid network problems that cause the interruption of use case execution and improve the completion rate of use case execution. If the device network is abnormal and the differential package or the complete package cannot be downloaded normally, access the installation package storage service through the wired network, download the package to the local computer, and establish a local warehouse, and then push the package to the specified directory of the device through ADB push. Finally, trigger the quickUpgrade service through the ADB broadcast command for silent upgrade.

[0154] Step C: Obtain the log file generated by the APP through ADB commands and query key log information.

[0155] Based on the log information, determine whether the APP performs differential package upgrade or pulls the full package. Combined with step B, determine whether the APP is executed correctly. If the differential package is downloaded successfully, but the differential package upgrade is not executed, the execution is abnormal and the hot fix failure information 3 is recorded. Therefore, the differential package upgrade execution is abnormal, which will not cause the upgrade to be interrupted, so continue to the next step.

[0156] Step D: Use ADB command to poll the specified directory / install within a certain period of time to check whether the APK installation package exists. If the APK file does not exist within the specified time, according to step C, based on the actual execution action, determine whether the hot repair fails or the download of the full package fails, and record the hot repair failure information 4. Otherwise, proceed to the next step of verification.

[0157] Step E: Obtain the log file generated by the App through the ADB command, and request the backend interface to obtain the current version information. Compare the version information obtained from the two channels. If they are consistent, the upgrade is successful. If they are inconsistent, record the hot fix failure information 5.

[0158] Step F: Use ADB commands to detect the changes in the PID and running memory during the APP operation within a period of time to determine whether the APP is running normally after the upgrade. If the PID does not change and the running memory reaches the normal running memory, the upgrade is successful. Otherwise, the upgrade fails and the hot repair failure information 6 is recorded.

[0159] By verifying and monitoring each link of the hot fix, accurately locating the failed link, and providing effective environmental information and log information, the efficiency of problem troubleshooting is improved and the process of reproducing bugs is reduced. And through detailed test reports, the abnormal links are clear at a glance, which helps to quickly locate the problem. For example, the problem of abnormal download caused by network problems can be automatically troubleshooted and located without manual intervention. By refining the verification link and simulated upgrade link, as well as network detection (continuous ping of the public network IP), the upgrade failure caused by network factors can be automatically eliminated. For other abnormal problems, the abnormal link and execution steps, as well as detailed environmental information and log information, can be provided to improve the efficiency of problem troubleshooting and reduce the process of reproducing bugs.

[0160] In the test report stage, reports are generated based on the use case and each test node, and the upgrade failure analysis is recorded to output relatively accurate problem descriptions, such as network anomalies, failure to synthesize differential packets into complete packets, hot loading anomalies, etc., which can help companies reduce costs and increase efficiency and improve product quality.

[0161] Another embodiment of the present invention provides an automatic testing device for thermal repair, such as Figure 6 As shown, the device 1 comprises:

[0162] A decoupling module 11, used for decoupling the test case and the execution script into a test case file and a test program;

[0163] The parsing module 12 is used to parse the test case file through the test program and generate a test set for automatic testing when the device to be tested is connected;

[0164] The communication and control module 13 is used to connect to multiple devices to be tested at the same time and allocate an independent automated test task thread to each test device;

[0165] The automated testing module 14 is used to perform automated hotfix testing according to each upgrade scenario in the test case file and generate test results;

[0166] The report generating module 15 is used to generate a test report according to the test result.

[0167] The modules described in the present invention refer to a series of computer program instruction segments that can perform specific functions. These modules are more suitable for describing the execution of the test data automatic generation process. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.

[0168] When writing test cases, YAML language is used. This embodiment decouples test cases and execution scripts into YAML case files and test programs. In this way, testers only need to understand YAML language. Technical personnel only need to write YAML case files according to test requirements and adopt the protocol agreed with the test program. It has low requirements on coding ability and test case writing can be completed without coding, reducing the difficulty of case writing and maintenance costs.

[0169] In one embodiment, the parsing module 12 includes:

[0170] A first parsing unit, configured to parse the test case file through the test program, wherein the test case is written in yaml language;

[0171] Test set generation, used to generate json test set from parsed test case files;

[0172] The second parsing unit is used to parse the JSON test set for automatic testing when the device to be tested is connected.

[0173] In one embodiment, the communication and control module 13 includes:

[0174] A communication unit, used to simultaneously connect multiple devices to be tested via a USB interface and / or WiFi;

[0175] The control unit is used to dynamically allocate a process identifier to each device to be tested; and to create an independent automated test task thread for each device to be tested according to the process identifier to realize concurrent testing.

[0176] In one embodiment, the automated testing module 14 includes:

[0177] A configuration unit, configured to configure test tasks according to various upgrade scenarios, wherein the upgrade scenarios include but are not limited to user ID upgrade, hot fix ID upgrade, full upgrade, specified version upgrade, abnormal crash handling, download package deletion, and upgrade package replacement;

[0178] The verification unit is used to verify the hot repair status and progress through backend interface status query, real-time Android log query and system information ADB query during hot repair;

[0179] The processing unit is used to generate corresponding test results when it is found that the automated test is completed.

[0180] In one embodiment, the verification unit includes:

[0181] The query subunit is used to poll the backend interface to obtain the hot repair push time after the hot repair task is triggered;

[0182] The first verification subunit is used to detect the download status of the differential package through ADB instructions when the hot fix push time is obtained, and to verify whether the differential package download is successful;

[0183] The second check subunit is used to obtain key log information in the log file generated by the hot fix program through ADB instructions when the differential package is downloaded successfully, and determine whether the hot fix program performs a differential package upgrade or a full package upgrade during hot fix according to the key log information;

[0184] The third verification subunit is used to detect whether the full upgrade installation package exists through ADB instructions when identifying that the hot fix program executes a full package upgrade;

[0185] The judgment subunit is used to detect the thread identification change and running memory when the hot fix program is running through ADB instructions when the full upgrade installation package is detected, so as to judge whether the program runs normally after the upgrade.

[0186] The second check subunit is also used to identify the failure of the hot fix differential upgrade and record the failure information when the differential package is downloaded successfully but the hot fix executes the full package upgrade;

[0187] The third verification subunit is also used to identify the failure of the hot fix full package upgrade and record the failure information when it is detected that the full upgrade installation package does not exist.

[0188] Since the functions and features of each module in the automatic testing device have been described in detail in the automatic testing method above, they will not be repeated here.

[0189] Another embodiment of the present invention provides an automatic test device for thermal repair, such as Figure 7 As shown, the device 10 includes:

[0190] One or more processors 110 and memory 120, Figure 7 A processor 110 is used as an example for description. The processor 110 and the memory 120 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0191] The processor 110 is used to complete various control logics of the device 10, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISCMachine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0192] The memory 120 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the automatic testing method for hot repair in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, implementing the automatic testing method for hot repair in the above method embodiment.

[0193] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the device 10, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0194] One or more units are stored in the memory 120, and when executed by one or more processors 110, the steps in the above automatic testing method are implemented:

[0195] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the above steps S100 to S500 are implemented.

[0196] As an example, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0197] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, a floppy disk, a flash memory, an optical storage device, etc.

[0198] Compared with the prior art, the present invention decouples test cases and execution scripts, supports the simultaneous development of test cases and execution scripts, and improves collaborative efficiency. During testing, multiple devices are accessed, and different hardware devices are supported to be tested simultaneously, so as to achieve efficient execution of compatibility testing and double the efficiency.

[0199] When performing automated testing after hot fixes, accurate verification is performed through interfaces, logs, and system information to achieve unattended automated testing.

[0200] In addition, by writing test cases in the YAML language, testers do not need to learn anything extra, which reduces the company's personnel costs and can efficiently complete hot fix testing tasks.

[0201] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. An automatic testing method for hot repair, characterized in that: include: Decouple test cases and execution scripts into test case files and test programs; Parsing the test case file through the test program and generating a test set for performing automated testing when the device to be tested is connected; Connect to multiple devices to be tested at the same time and assign independent automated test task threads to each test device; Performing hot fix automation testing according to each upgrade scenario in the test case file and generating test results; A test report is generated according to the test results.

2. The automatic testing method for thermal repair according to claim 1, characterized in that: The step of parsing the test case file by the test program and generating a test set for performing automated testing when the device to be tested is connected includes: Parsing the test case file by the test program, wherein the test case is written in yaml language; Generate a json test set based on the parsed test case file; The JSON test set is parsed for automated testing when the device to be tested is connected.

3. The automatic testing method for thermal repair according to claim 1, characterized in that: The step of simultaneously connecting to a plurality of devices to be tested and allocating an independent automated test task thread to each test device comprises: Connect multiple devices to be tested simultaneously through USB interface and / or WiFi, and dynamically assign process ID to each device to be tested; An independent automated test task thread is created for each device to be tested according to the process identifier to implement concurrent testing.

4. The automatic testing method for thermal repair according to claim 1, characterized in that: The step of performing hot fix automation testing according to each upgrade scenario in the test case file and generating test results includes: Configure test tasks according to various upgrade scenarios, including but not limited to user ID upgrade, hotfix ID upgrade, full upgrade, specified version upgrade, abnormal crash handling, download package deletion, and upgrade package replacement; During hot fixes, verify the hot fix status and progress through backend interface status query, real-time Android log query, and system information ADB query; When the automated test is completed, the corresponding test results are generated.

5. The automatic testing method for thermal repair according to claim 4, characterized in that: During the hot repair, the steps of verifying the hot repair status and progress through backend interface status query, real-time Android log query and system information ADB query include: After the hot fix task is triggered, poll the backend interface to obtain the hot fix push time; When the hot fix push time is obtained, use ADB commands to check the download status of the differential package and verify whether the differential package download is successful; When the differential package is downloaded successfully, the key log information in the log file generated by the hot fix program is obtained through the ADB command, and according to the key log information, it is determined whether the hot fix program performs a differential package upgrade or a full package upgrade during the hot fix; When a hotfix is ​​identified and a full package upgrade is performed, the ADB command is used to detect whether the full upgrade installation package exists; When a full upgrade installation package is detected, the thread ID changes and running memory during the operation of the hot fix program are detected through ADB commands, and whether the program runs normally after the upgrade is determined based on the thread ID changes and running memory.

6. The automatic testing method for thermal repair according to claim 5, characterized in that: During the hot repair, the step of verifying the hot repair status and progress through backend interface status query, real-time Android log query and system information ADB query also includes: When the differential package is downloaded successfully, but the hot fix performs a full package upgrade, the hot fix differential upgrade failure is identified and the failure information is recorded; When it is detected that the full upgrade installation package does not exist, it is identified that the hot fix full package upgrade has failed, and the failure information is recorded.

7. The automatic testing method for thermal repair according to claim 5, characterized in that: The step of performing hot fix automation testing according to each upgrade scenario in the test case and generating test results also includes: Use ADB commands to obtain the hot fix version information in the log file generated by the hot fix, and use the backend interface to obtain the current version information of the hot fix, and compare the two version information to see if they are the same; When the two version information are the same, it is identified that the hot fix program is successfully upgraded; when the two version information are different, it is identified that the hot fix program fails to be upgraded, and the failure information is recorded.

8. An automatic test device for thermal repair, characterized in that: include: Decoupling module, used to decouple test cases and execution scripts into test case files and test programs; A parsing module, used to parse the test case file through the test program and generate a test set for automatic testing when the device to be tested is connected; The communication and control module is used to connect to multiple devices to be tested at the same time and allocate an independent automated test task thread to each test device; An automated testing module, used to perform automated hot fix testing according to each upgrade scenario in the test case file and generate test results; A report generating module is used to generate a test report according to the test results.

9. An automatic test device for thermal repair, characterized in that: The device comprises at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the automatic testing method for thermal repair according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the automatic testing method for thermal repair according to any one of claims 1 to 7.

Images