Test method, device and chip for application program

Abstract

The application relates to a test method, device and chip of an application program. The method comprises the following steps: obtaining a tracking file set of display information in a process in which a target application program runs a test case through a track tracking tool; obtaining sliding operation information corresponding to the test case and a consumption quantity of a graphic buffer from the tracking file set; determining a test index of running fluency of the target application program according to the sliding operation information and the consumption quantity of the graphic buffer; and determining the running fluency of the target application program according to the test index. The method can improve the accuracy of a test result.

Description

Technical Field

[0001] This application relates to the field of software testing technology, and in particular to a testing method, apparatus and chip for an application. Background Technology

[0002] Application smoothness testing is a crucial part of application performance testing. It focuses on detecting the user experience and visual performance of the application throughout its entire lifecycle, determining whether there are any stuttering issues during application operation, and ultimately ensuring that the application can stably reach the frame rate standard when running on electronic devices, ensuring responsive operation and smooth display.

[0003] In related technologies, the smoothness test results of an application are usually determined based on images during application execution. However, this method suffers from the problem of inaccurate test results. Summary of the Invention

[0004] Therefore, it is necessary to provide a testing method, apparatus, and chip for applications that can improve the accuracy of test results, addressing the aforementioned technical problems.

[0005] Firstly, this application provides a method for testing an application, comprising:

[0006] Use a trajectory tracing tool to obtain a set of tracing files that display information during the execution of test cases in the target application;

[0007] Obtain the sliding operation information and the amount of graphics buffer consumed corresponding to the test case from the tracking file set;

[0008] Based on the sliding operation information and the amount of graphics buffer consumed, a test metric for the smoothness of the target application's operation is determined.

[0009] The smoothness of the target application's operation is determined based on the test metrics.

[0010] In one embodiment, the sliding operation information includes the sliding time; determining the test metric for the smoothness of the target application's operation based on the sliding operation information and the consumption of the graphics buffer includes: determining the frame rate information during the operation of the target application based on the sliding time and the consumption of the graphics buffer; determining the number of frame drops and the maximum number of consecutive frame drops during the operation of the target application based on the consumption of the graphics buffer; and determining the frame rate information, the number of frame drops, and the maximum number of consecutive frame drops as the test metric.

[0011] In one embodiment, determining the number of frame drops and the maximum number of consecutive frame drops during the operation of the target application based on the consumption of the graphics buffer includes: determining the valid frame drop events and consecutive frame drop events that occur during the operation of the target application based on the consumption of the graphics buffer; determining the number of frame drops based on the number of valid frame drop events; and determining the maximum number of consecutive frame drops from the consecutive frame drop events.

[0012] In one embodiment, determining the valid frame drop events and consecutive frame drop events occurring during the operation of the target application based on the consumption of the graphics buffer includes: determining the frame drop events and consecutive frame drop events occurring during the operation of the target application based on the consumption of the graphics buffer; identifying a plurality of consecutive frame drop events as first valid frame drop events; identifying all first frame drop events and second frame drop events occurring during the operation of the target application as valid frame drop events; the second frame drop event is a frame drop event other than the first frame drop event.

[0013] In one embodiment, obtaining the sliding operation information and the consumption quantity of the graphics buffer corresponding to the test case from the trace file set includes: identifying abnormal trace files in the trace file set; removing the abnormal trace files to obtain a trace file set after removal; and obtaining the sliding operation information and the consumption quantity of the graphics buffer from the trace file set after removal.

[0014] In one embodiment, determining the abnormal tracking file in the tracking file set includes: acquiring a first image of the interface displayed before the target application runs each operation in the test case, and a second image of the interface displayed after running each operation; determining a target operation corresponding to a difference value between the first image and the second image that is less than a preset threshold; and determining the tracking file corresponding to the target operation as the abnormal tracking file.

[0015] In one embodiment, the method further includes: updating the trace file set according to the test metric or the consumption quantity of the graphics buffer; obtaining the updated sliding operation information and the updated consumption quantity of the graphics buffer corresponding to the test case from the updated trace file set; and updating the test metric according to the updated sliding operation information and the updated consumption quantity of the graphics buffer.

[0016] In one embodiment, updating the tracking file set based on the test metric or the consumption of the graphics buffer includes any one of the following: determining the average consumption of the graphics buffer and updating the tracking file set based on the average; updating the tracking file set based on the maximum number of consecutive dropped frames; or updating the tracking file set based on the frame rate information.

[0017] Secondly, this application also provides a testing apparatus for an application, comprising:

[0018] The first acquisition module is used to acquire the set of trace files that display information during the execution of test cases in the target application through a trajectory tracing tool;

[0019] The second acquisition module is used to acquire the sliding operation information and the consumption amount of the graphics buffer corresponding to the test case from the tracking file set;

[0020] The first determining module is used to determine the test index of the running smoothness of the target application based on the sliding operation information and the consumption of the graphics buffer.

[0021] The second determining module is used to determine the running smoothness of the target application based on the test indicators.

[0022] Thirdly, this application also provides a terminal device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect above.

[0023] Fourthly, this application also provides a chip including a processor and a communication interface, the processor being configured to cause the chip to perform the method described in the first aspect above.

[0024] Fifthly, this application also provides a chip module, including a communication module, a power module, a storage module, and a chip, wherein:

[0025] The power module is used to provide power to the chip module;

[0026] The storage module is used to store data and instructions;

[0027] The communication module is used for internal communication within the chip module, or for communication between the chip module and external devices.

[0028] The chip is used to perform the steps of the method provided in the first aspect above.

[0029] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0030] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0031] The aforementioned testing methods, apparatus, terminal devices, chips, chip modules, computer-readable storage media, and computer program products acquire a set of trace files containing information displayed during the execution of test cases in the target application using a trajectory tracing tool. From the trace file set, they obtain sliding operation information and the amount of graphics buffer consumption corresponding to the test cases. Based on the sliding operation information and the amount of graphics buffer consumption, they determine test metrics for the smoothness of the target application's operation. Based on the test metrics, they determine the smoothness of the target application's operation. Therefore, this embodiment of the application simultaneously acquires trace files during the execution of test cases in the target application, parses the two core data points—sliding operation information and the amount of graphics buffer consumption—from the trace files, automatically determines test metrics based on this data, and determines the smoothness test results based on the test metrics. Compared to methods that test based on images during application execution, this approach improves test accuracy, thereby improving the accuracy of the test results. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is an internal structure diagram of a terminal device in one embodiment;

[0034] Figure 2 This is a logical diagram of the software architecture in one embodiment;

[0035] Figure 3 This is a flowchart illustrating the testing method for an application in one embodiment;

[0036] Figure 4 This is a schematic diagram of a tracking file in one embodiment;

[0037] Figure 5 This is a flowchart illustrating step 303 in one embodiment;

[0038] Figure 6 This is a flowchart illustrating step 302 in one embodiment;

[0039] Figure 7 This is a flowchart illustrating the testing method for an application in another embodiment;

[0040] Figure 8 This is a flowchart illustrating the testing method of an application in an example.

[0041] Figure 9 This is a structural block diagram of a testing device for an application in one embodiment;

[0042] Figure 10 This is an internal structure diagram of a chip module in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0045] In related technologies, application smoothness testing either relies on high-speed cameras and image recognition algorithms, which result in inaccurate test results, expensive equipment, and the inability to achieve full automation, large-scale testing, and the need for retesting when anomalies are found; or it relies on image comparison modules to compare two images, which involves a high workload or algorithm complexity, depends on the accuracy of image recognition algorithms, and is subject to visual persistence effect, leading to inaccurate test results.

[0046] Therefore, this application provides a testing method for an application, which automates the testing of application smoothness, replaces the testing methods in related technologies, can perform a large number of tests, and detect and remove abnormal data, solves the impact of occasional performance problems on the results, and reduces the workload of manual confirmation; the test results can provide a lot of maintenance and testing information to assist R&D in locating problems.

[0047] The embedded system startup method provided in this application embodiment can be applied to terminal devices, and its internal structure diagram can be as follows: As shown, the device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external devices; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a test method for an application. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the terminal device, or external keyboards, touchpads, or mice, etc.

[0048] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the terminal device to which the solution of this application is applied. A specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0049] It should be noted that the testing method in this application embodiment can be achieved through... Figure 2 The software architecture diagram shown is an implementation of the software architecture. The software architecture includes a resource provisioning module, a test case library, an execution module, a maintenance and testing module, a parsing module, and a reporting module.

[0050] Resource provisioning module: Responsible for pre-test preparation, including the installation, compilation, and optimization of the target application.

[0051] Test case library: Internally stores test case designs for test scenarios (such as sliding scenarios). This includes the target application, specific interface, preconditions, test steps, and test standards for the sliding scenarios.

[0052] Execution Module: Responsible for the automated execution of test cases in the test case library, including sliding execution, pop-up removal, etc. The execution module includes adb command encapsulation and UIAutomator2 function encapsulation.

[0053] Dimensional Monitoring Module: Responsible for collecting dimensional monitoring information before, during, or after scrolling, used for frame rate parsing, anomaly detection, and assisted localization. Outputs the following information:

[0054] Trace file: A trace file generated during the sliding process, used for frame rate analysis and problem localization; trace supports the systrace and perfetto tools;

[0055] Images: Screenshots before and after the swipe, used to determine whether the scene is abnormal and whether the swipe process is executed normally;

[0056] SurfaceFlinger service state dump (Dumpsys sur): Grabs data before and after swiping, providing auxiliary information for positioning;

[0057] Package Management Service Status Dump (Dumpsys pac): Slides forward and backward to capture auxiliary information for location purposes;

[0058] Graphics rendering information dump (Dumpsys gfx): Captures information before and after sliding, providing auxiliary information for positioning;

[0059] Device info: Captured before testing for identification purposes.

[0060] Parsing module: Responsible for parsing the tracking file captured during the sliding process, and performing metric statistics and outlier removal. Metrics include frame rate, number of dropped frames, and maximum number of consecutive dropped frames.

[0061] Reporting module: Visualizes the results output by the analysis module, presenting statistical test metrics in tabular form.

[0062] In one exemplary embodiment, such as Figure 3 As shown, a testing method for an application is provided, which is applied to... Figure 1 The following steps, 301 to 304, are used as examples of terminal devices or chips / chip modules with data processing capabilities. Specifically:

[0063] Step 301: Obtain the set of trace files that display information during the execution of test cases in the target application using a trajectory tracing tool.

[0064] Tracing tools are system-level performance profiling tools that synchronously record the timing event flow within the operating system during program runtime and generate timestamped trace files. Examples of tracing tools include systrace and perfetto. The target application and program refer to the application whose smoothness of operation needs to be tested. Test cases are pre-defined cases used to test the smoothness of the target application's operation based on actual needs and historical experience. There can be multiple test cases. Each application to be tested has its own test cases, all stored in a test case library. The test case library includes multiple test cases, each containing test scenario information (e.g., a swipe scenario), specifically including the target application's identifier, the specific interface, preconditions, test steps, and test standards.

[0065] For example, when a terminal device needs to test the smoothness of its application or receives a smoothness test instruction, it calls the test case library and triggers the target application to run its test cases in sequence according to multiple test cases in the test case library. Each test case can be a swipe operation, and during each swipe operation, a trajectory tracing tool is used to capture the trace file of the display information. After multiple swipes, a set of trace files of the display information is obtained. Each trace file corresponds to a swipe operation of the target application by the execution model, such as swiping the status bar, swiping the task list, and swiping the settings list.

[0066] Each tracking file corresponds to one image rendering timing event for a swipe operation. As an example, based on the phone's image rendering process, the content of the tracking file corresponding to a swipe operation is as follows: Figure 4 As shown, where, Figure 4 The horizontal axis represents the time axis, with each vertical sync signal (vsync-sf) cycle lasting approximately 16.6ms. The vertical axes represent the touch screen signal, the application vertical sync signal (vsync-app), the system synthesized vertical sync signal (vsync-sf), and the number of images to be processed in the image buffer queue (bufferTx).

[0067] Touchscreen signal: Value (0 or 1), 1 represents a touchscreen signal, 0 represents no signal; it appears multiple times in the process because touchscreen signals will occur multiple times during a single swipe. The system also uses these signals to identify gesture behavior, such as the swipe process in this case.

[0068] vsync-sf, vsync-app: Value (0 or 1). The value has no meaning. Switching between 0 and 1 or between 1 and 0 indicates that a signal has occurred, triggering the task to execute.

[0069] bufferTx: A numerical value representing the number of buffers. One buffer will be added after the application is drawn and rendered, and one buffer will be consumed when the system takes it away for compositing. If no image buffer is consumed within a vsync-sf cycle, it indicates a frame drop.

[0070] Step 302: Obtain the sliding operation information and the amount of graphics buffer consumption corresponding to the test cases from the tracking file set.

[0071] The sliding operation information refers to the sliding-related information during the sliding execution of a test case, including at least the sliding action. The graphics buffer consumption refers to the total number of vsync compositing cycles (beats) successfully retrieved by the system compositor (SurfaceFlinger) and used for screen display corresponding to one sliding operation.

[0072] For example, after obtaining the set of tracking files, the parsing module automatically parses all tracking files to obtain the sliding operation information and the amount of image buffer consumption corresponding to each tracking file. Optionally, the sliding interval is first determined based on the parsing results, and then the number of events in the image buffer where the system compositor successfully consumes the image buffer is determined within the sliding interval, thus obtaining the amount of image buffer consumption.

[0073] Step 303: Determine the test metrics for the smoothness of the target application's operation based on the sliding operation information and the amount of graphics buffer consumption.

[0074] For example, the terminal device determines test metrics for the smoothness of the target application's operation based on each set of swipe operation information and the amount of graphics buffer consumption, thus generating multiple sets of test metrics. Optionally, the swipe operation information includes at least the swipe time, the ratio between the amount of graphics buffer consumption and the swipe time is calculated, the frame rate information during the operation of the target application is determined based on the ratio, and this frame rate information is used as a test metric for the smoothness of operation.

[0075] Step 304: Determine the smoothness of the target application's operation based on the test metrics.

[0076] For example, after obtaining multiple sets of test metrics, the smoothness of the target application is comprehensively evaluated based on all test metrics to obtain a smoothness test result. Optionally, the smoothness (or smoothness level) of the target application is determined based on the average frame rate information of each test metric.

[0077] It should be noted that the testing method in this application embodiment may vary slightly during the automated execution process for different terminal device forms. For example, in the automated execution of swiping operations, different swiping coordinates are required for mobile phones, tablets, watches, and in-vehicle systems due to their different forms. Since the graphics display architecture on the Android system is consistent, the method for parsing indicators is also consistent. If the display architecture is adjusted due to Android system upgrades, the corresponding modifications can be made to the adjustment points. The differences in scenario design and swipe command execution vary depending on the form factor of the terminal device. For example, smartphones are portrait-oriented devices; smart tablets are landscape-oriented, and may have secondary menus in their settings interface, so different swipe interfaces need to be designed for specific scenarios in the test case design. When executing commands, it is necessary to consider the differences from mobile phone commands. For example, the starting point of an upward swipe on a mobile phone can be set to 1 / 2 horizontally and 3 / 4 vertically, and the ending point can be 1 / 2 horizontally and 1 / 4 vertically. For a tablet, it is necessary to consider whether this coordinate system is applicable. For the use of secondary menus, it is necessary to consider whether the 1 / 2 horizontal coordinate falls on the first or second level. Smartwatches: Test cases for smartwatch scenarios need to be redesigned for specific scenarios. The swipe coordinates are consistent with those of tablets, but debugging is required for specific scenarios. In-vehicle systems: In-vehicle systems are generally landscape-oriented, and the specific interface display will also be different, requiring separate test scenario design. The coordinate design scheme is the same, but debugging is required for specific scenarios.

[0078] In the aforementioned application testing method, a set of trace files containing displayed information during the execution of test cases in the target application is obtained using a trajectory tracing tool; the sliding operation information and the consumption of the graphics buffer corresponding to the test cases are obtained from the trace file set; based on the sliding operation information and the consumption of the graphics buffer, a test metric for the smoothness of the target application is determined; and based on the test metric, the smoothness of the target application is determined. Therefore, this embodiment of the application synchronously obtains trace files during the execution of test cases in the target application, parses out the two core data points—sliding operation information and the consumption of the graphics buffer—from the trace files, automatically determines the test metric based on this data, and determines the smoothness test result based on the test metric. Compared to methods that test based on images during application execution, this method improves test accuracy, thereby improving the accuracy of the test results.

[0079] The following example illustrates how to determine test metrics.

[0080] In one exemplary embodiment, the sliding operation information includes the sliding time. For example... Figure 5 As shown, step 303 includes steps 501 to 503. Wherein:

[0081] Step 501: Determine the frame rate information during the execution of the target application based on the sliding time and the amount of graphics buffer consumed.

[0082] For example, firstly, the sliding interval is determined: refer to Figure 4 The swipe process is divided into two phases: follow-up and release. During the follow-up phase, an App Queue (aq) event occurs and ends. The release phase simulates the inertial movement of the interface after the finger leaves the screen; that is, no more frame refreshes occur after the aq event ends. Then, the swipe time is determined: after determining the swipe interval including both the follow-up and release phases, the sum of the times for these two phases is analyzed, which is the swipe time corresponding to this operation. Next, the amount of graphics buffer consumed is determined: during the swipe process, based on... Figure 4 The rendering flow shown demonstrates that the display framework revolves around an intermediate graphics buffer. The target application is responsible for generating the buffer in ticks (1s / 60, 16.6ms for 60Hz). The compositor (SurfaceFlinger) is responsible for consuming the buffer in ticks to composite the content needed for the display screen. Therefore, the parsing module can obtain the amount of buffer consumed during the sliding time by parsing the tracking file. If no buffer is consumed in a certain tick, frame drops occur, and the frame rate decreases. The maximum consecutive frame drops can be determined by counting multiple consecutive ticks without buffer consumption. Finally, the frame rate information is determined based on the ratio between the amount of graphics buffer consumed and the sliding time. Optionally, the frame rate for this sliding operation can be calculated using the following formula: Frame rate = Amount of graphics buffer consumed / Sliding time.

[0083] Step 502: Based on the amount of graphics buffer consumed, determine the number of frame drops and the maximum number of consecutive frame drops during the execution of the target application.

[0084] For example, for each tracking file, the entire sliding window's vsync cycles are determined using the system's vertical sync signal (vsync) as the beat (e.g., every 16.6ms as a cycle). Within each vsync cycle, it is determined whether a graphics buffer consumption event has occurred (i.e., whether the system compositor has performed a display operation). If no graphics buffer consumption event occurs within a certain cycle, it is determined that a frame drop event has occurred in that cycle. The total number of all frame drop events is the frame drop count. Among all frame drop events, the longest consecutive frame drop event is selected, and the number of frame drop events corresponding to this consecutive frame drop event is determined as the maximum consecutive frame drop count.

[0085] Step 503: Frame rate information, number of frame drops, and maximum number of consecutive frame drops are determined as test indicators.

[0086] For example, test cases, sliding operations, and trace files have a one-to-one correspondence. The frame rate information, number of dropped frames, and maximum number of consecutive dropped frames obtained from each trace file are used as a set of test metrics, resulting in multiple sets of test metrics. The average frame rate information, the average number of dropped frames, and the maximum number of consecutive dropped frames in each set of test metrics are determined and used as the final test metric. The test result for the smoothness of the target application's operation is determined based on the maximum value among the average frame rate, average number of dropped frames, and maximum number of consecutive dropped frames.

[0087] Optionally, the number of test cases should not be less than the preset number, such as 10, to ensure sufficient test metrics and guarantee test reliability.

[0088] Therefore, in this embodiment, the frame rate, number of frame drops, and maximum number of consecutive frame drops during the operation of the target application are determined based on the sliding time and the amount of graphics buffer consumption, which serves as the sliding index and ensures the completeness and accuracy of the index.

[0089] In an exemplary embodiment, step 502 includes: determining valid frame drop events and consecutive frame drop events that occur during the operation of the target application based on the amount of graphics buffer consumption; determining the number of frame drops based on the number of valid frame drop events; and determining the maximum number of consecutive frame drops from the consecutive frame drop events.

[0090] For example, a valid frame drop event can be a subset of all the frame drop events identified above, or it can be a subset of all frame drop events after validity checks, merging, and other processing. The number of valid frame drop events is less than the number of frame drop events.

[0091] For example, based on the amount of image buffer consumed, all the parsed frame drop events are subjected to validity checks or merged to obtain valid frame drop events. The number of valid frame drop events is used to determine the number of non-frame drop events. Then, based on the number of consecutive beats that do not consume the image buffer, consecutive frame drop events are determined. From at least one consecutive frame drop event (each consecutive frame drop event includes multiple consecutive frame drop events), the consecutive frame drop event with the most frame drop events is determined, and the number of this consecutive frame drop event is determined as the maximum number of consecutive frame drop events. Based on the number of consecutive beats that do not consume the image buffer, the maximum number of consecutive frame drop events can be calculated.

[0092] Therefore, this embodiment determines the number of frame drops based on the number of valid frame drop events, and determines the maximum number of consecutive frame drops from consecutive frame drop events, which can ensure the effectiveness of the indicator statistics.

[0093] In an exemplary embodiment, determining valid frame drop events and consecutive frame drop events occurring during the execution of the target application based on the consumption of the graphics buffer includes: determining frame drop events and consecutive frame drop events occurring during the execution of the target application based on the consumption of the graphics buffer; identifying multiple consecutive frame drop events as first frame drop events; identifying all first frame drop events and second frame drop events occurring during the execution of the target application as valid frame drop events; the second frame drop event is a frame drop event other than the first frame drop event.

[0094] In this context, a first frame drop event refers to a group of consecutive frame drop events. Multiple consecutive frame drop events are merged into a single first frame drop event; that is, the first frame drop event is a merged event. Frame drop events that are not merged are called second frame drop events. All first frame drop events and all second frame drop events constitute all valid frame drop events.

[0095] For example, the number of frame drops is determined based on whether the graphics buffer is consumed in each beat. Multiple consecutive frame drops without a graphics buffer are merged into a first frame drop event, and frame drops that fail to be merged are considered second frame drops. The first and second frame drops are considered as valid frame drops, and the total number of valid frame drops is the number of frame drops.

[0096] Therefore, this embodiment merges multiple consecutive frame drop events that do not consume the graphics buffer into one algorithm, which can more realistically simulate human operation and more accurately reflect the number of stutters caused by smoothness issues, thereby ensuring the accuracy of the test.

[0097] It should be noted that, in order to further improve the accuracy of the test, in order to obtain effective and stable data in static scene testing, it is necessary to exclude some outliers. Outliers include at least one of the following: unsuccessful sliding operation, frame rate fluctuations, and network environment-related fluctuations. Examples are provided below:

[0098] That is, in an exemplary embodiment, such as Figure 6 As shown, step 302 includes steps 601 to 603, wherein:

[0099] Step 601: Identify the abnormal tracking files in the tracking file set.

[0100] Step 602: Remove the abnormal tracking files to obtain the set of removed tracking files.

[0101] Step 603: Obtain the sliding operation information and the amount of graphics buffer consumption from the removed tracking file set.

[0102] For example, data validation can be performed on the data in each tracking file. If the data is invalid, blank, or abnormal, the tracking file is identified as an abnormal tracking file and removed. If the data is not invalid, blank, or abnormal, the tracking file is identified as a normal tracking file and retained, thus obtaining a set of removed tracking files. Each removed tracking file is then parsed by the parsing module to obtain the sliding operation information and the consumption amount of the graphics buffer.

[0103] Therefore, this embodiment can ensure the validity of the statistical data of the indicators from the source by removing outliers, and can ensure the reliability of the indicators, thereby ensuring the reliability of the test.

[0104] In an exemplary embodiment, step 601 includes: acquiring a first image of the interface displayed before each operation in the test case of the target application, and a second image of the interface displayed after each operation; determining the target operation corresponding to the difference between the first image and the second image being less than a preset threshold; and determining the trace file corresponding to the target operation as an abnormal trace file.

[0105] For example, the side-view module captures an image of the display interface before and after each sliding operation to obtain a first image and a second image of the display interface. The two images are compared using an image recognition algorithm to obtain the difference value between the two images. If the difference value is less than a preset threshold, the content of the two images is determined to be consistent. The sliding operation is identified as the target operation, and the tracking file corresponding to the target operation is identified as an abnormal tracking file. It is not parsed or statistically analyzed, and is stored in a separate folder.

[0106] Therefore, the abnormal tracking file is identified based on the screenshot image and then deleted, ensuring the reliability of the data involved in the indicator analysis. Moreover, the method for identifying the abnormal tracking file is simple and easy to implement.

[0107] After the number of test metrics or graph buffers has been consumed, the tracking file set can be updated to further improve the reliability of the tracking file set based on the above-mentioned elimination.

[0108] That is, in an exemplary embodiment, such as Figure 7 As shown, the method further includes steps 701 to 703, wherein:

[0109] Step 701: Update the tracking file set based on the consumption of the test metrics or the graph buffer.

[0110] For example, the tracking file set can be updated based on frame rate information, the maximum number of consecutive dropped frames, or the amount of graphics buffer consumption.

[0111] Optionally, the number of graphics buffers consumed for each slide is the total number of frames for that slide. This value reflects the total duration of the slide. Under normal circumstances, the total number of frames for each slide should be roughly the same, within the same order of magnitude. When an anomaly occurs, the total number of frames may be too large or too small. If it is too large, the impact on data statistics is relatively small, while if it is too small, the impact on the results is large. Therefore, the tracking files corresponding to the excessively large or small total number of frames are deleted from the tracking file set to update the tracking file set.

[0112] Optionally, for each maximum consecutive frame loss corresponding to each slide, the maximum consecutive frame loss can be compared and verified, and the abnormal maximum consecutive frame loss can be selected. The tracking file corresponding to the abnormal maximum consecutive frame loss can be removed from the tracking file set to update the tracking file set.

[0113] Optionally, for all frame rate information corresponding to each slide, the frame rates can be compared and verified, and abnormal frame rates can be selected. The tracking files corresponding to the abnormal frame rates can be removed from the tracking file set to update the tracking file set.

[0114] Step 702: Obtain the updated sliding operation information and the updated consumption amount of the graphics buffer corresponding to the test cases from the updated trace file set.

[0115] For example, for an updated trace file set, step 302 is performed to obtain the updated sliding operation information and the updated consumption amount of the graphics buffer corresponding to the test case from each updated trace file set.

[0116] Step 703: Update the test metrics based on the updated sliding operation information and the updated consumption of the graphics buffer.

[0117] For example, after obtaining the updated sliding operation information and the updated consumption of the graphics buffer, step 303 is executed to determine the test index of the running smoothness of the target application based on the updated sliding operation information and the updated consumption of the graphics buffer, thereby updating the test index.

[0118] Optionally, before performing the removal operation in this embodiment, it can be determined whether the number of currently tracked files is greater than or equal to a preset number, such as 10. If so, the removal operation is performed; otherwise, the removal operation is not performed.

[0119] Therefore, after parsing the tracking files or compiling statistics, the tracking file set is updated based on the parsing results or statistics, which further improves the reliability of the tracking files.

[0120] In one exemplary embodiment, step 701 includes any one of the following: determining an average number of graphics buffer consumptions and updating the tracking file set based on the average number; updating the tracking file set according to the maximum number of consecutive dropped frames; and updating the tracking file set according to the frame rate.

[0121] For example, tracking files with an abnormal total frame count are removed as follows: This assumes there are at least 10 tracking files. The average consumption of the image buffer is calculated, and tracking files with a consumption less than 0.3 times the average are identified as abnormal tracking files, as are those with a consumption greater than 0.6 times the average. Optionally, when removing multiple abnormal tracking files, the largest abnormal tracking file is removed each time, and the average is recalculated, with those less than 0.3 times the average being removed first.

[0122] Tracking file removal for abnormal maximum consecutive frame drop count: The prerequisite is that there are at least 10 tracking files. The judgment condition for abnormal maximum consecutive frame drop count is: there should be no isolated maximum value. For example, if the maximum consecutive frame drop count is 10, and there is another 10 or 9, then the tracking file corresponding to the maximum consecutive frame drop count of 10 is retained; if there is no other 10 or 9, then the tracking file corresponding to the maximum consecutive frame drop count of 10 is removed from the tracking file set. After removal, the current maximum consecutive frame drop count is recalculated until there are no isolated values.

[0123] Frame rate anomaly tracking file removal: The prerequisite is that there are at least 10 tracking files. The criteria for judging abnormal tracking files are: calculate the average and standard deviation of all frame rates, calculate the difference between each frame rate and the standard deviation, if the difference deviates from 1.5 times the standard deviation, the corresponding tracking file is considered an abnormal tracking file, remove the abnormal tracking file corresponding to the frame rate with the largest deviation, and recalculate until there are no abnormal frame rates.

[0124] Optionally, abnormal tracking files can be removed in the following order: unsuccessful sliding, abnormal total frame count, abnormal maximum consecutive frame drop, and abnormal frame rate. After this order is checked, an abnormal maximum consecutive frame drop count is checked again to ensure that there are no isolated values ​​for the maximum consecutive frame drop count.

[0125] Therefore, this embodiment updates the tracking file set based on the consumption of the graphics buffer, the maximum number of consecutive dropped frames, and the frame rate information, which can ensure the stability and effectiveness of the updated tracking file set.

[0126] The following describes a specific example of the testing method for the application of this application embodiment.

[0127] like Figure 8 As shown, the application testing method includes the following steps:

[0128] Step 801: Obtain the set of trace files that display information during the execution of test cases in the target application using a trajectory tracing tool;

[0129] Step 802: Identify the abnormal tracking files in the tracking file set;

[0130] Step 803: Remove the abnormal tracking files to obtain the set of removed tracking files;

[0131] Step 804: Obtain sliding operation information and the amount of graphics buffer consumption from the removed tracking file set;

[0132] Step 805: Determine the test metrics for the smoothness of the target application's operation based on the sliding operation information and the amount of graphics buffer consumption.

[0133] Step 806: Update the tracking file set based on the consumption of the test metrics or the graph buffer.

[0134] Step 807: Obtain the updated sliding operation information and the updated consumption amount of the graphics buffer corresponding to the test cases from the updated trace file set;

[0135] Step 808: Based on the updated sliding operation information and the updated consumption of the graphics buffer, update the test metrics to obtain the updated test metrics.

[0136] Step 809: Determine the smoothness of the target application's operation based on the updated test metrics.

[0137] In summary, the application testing method of this application embodiment calculates test metrics by automatically parsing trace files, verifies data validity by capturing maintenance information, and ensures data stability and validity through an outlier removal algorithm. Specifically, the resource pre-setting module is responsible for automatically pre-setting resources before sliding execution, the execution module automatically executes test steps, the maintenance module captures necessary information before and after the test process, the parsing module automatically parses the captured trace to obtain metrics such as the frame rate of interest and performs outlier detection, and the reporting module presents the output of the parsing module in a formatted table. This completes the calculation of the sliding frame rate and the construction of the testing system, and improves the accuracy of the test results.

[0138] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0139] Based on the same inventive concept, this application also provides a testing apparatus for an application that implements the testing method for the application described above. This apparatus can be applied to or integrated into a chip or chip module, for example. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more application testing apparatus embodiments provided below can be found in the limitations of the application testing method described above, and will not be repeated here.

[0140] In one exemplary embodiment, such as Figure 9 As shown, an application testing device is provided, comprising: a first acquisition module 901, a second acquisition module 902, a first determination module 903, and a second determination module 904, wherein:

[0141] The first acquisition module 901 is used to acquire a set of trace files that display information during the execution of test cases of the target application through a trajectory tracing tool;

[0142] The second acquisition module 902 is used to acquire the sliding operation information and the consumption amount of the graphics buffer corresponding to the test cases from the tracking file set;

[0143] The first determining module 903 is used to determine the test indicators of the running smoothness of the target application based on the sliding operation information and the amount of graphics buffer consumption.

[0144] The second determining module 904 is used to determine the smoothness of the target application's operation based on test metrics.

[0145] In one embodiment, the sliding operation information includes the sliding time; the first determining module 903 includes: a first determining unit, configured to determine the frame rate information during the operation of the target application based on the sliding time and the amount of graphics buffer consumption; a second determining unit, configured to determine the number of frame drops and the maximum number of consecutive frame drops during the operation of the target application based on the amount of graphics buffer consumption; and a third determining unit, configured to determine the frame rate information, the number of frame drops, and the maximum number of consecutive frame drops as test indicators.

[0146] In one embodiment, the second determining unit is specifically used to: determine valid frame dropping events and consecutive frame dropping events that occur during the operation of the target application based on the number of valid frame dropping events; determine the number of frame dropping events based on the number of valid frame dropping events; and determine the maximum number of consecutive frame dropping events from the consecutive frame dropping events.

[0147] In one embodiment, the second determining unit, when determining the valid frame drop events and consecutive frame drop events occurring during the operation of the target application based on the consumption of the graphics buffer, is specifically configured to: determine the frame drop events and consecutive frame drop events occurring during the operation of the target application based on the consumption of the graphics buffer; determine multiple consecutive frame drop events as first valid frame drop events; determine all first frame drop events and second frame drop events occurring during the operation of the target application as valid frame drop events; the second frame drop event is a frame drop event other than the first frame drop event.

[0148] In one embodiment, the second acquisition module 902 includes: a fourth determining unit, used to determine abnormal tracking files in the tracking file set; a removal unit, used to remove the abnormal tracking files to obtain a removed tracking file set; and an acquisition unit, used to acquire sliding operation information and the consumption amount of the graphics buffer from the removed tracking file set.

[0149] In one embodiment, the fourth determining unit is specifically used to: acquire a first image of the interface displayed before each operation in the test case of the target application, and a second image of the interface displayed after each operation; determine the target operation corresponding to the difference between the first image and the second image being less than a preset threshold; and determine the trace file corresponding to the target operation as an abnormal trace file.

[0150] In one embodiment, the apparatus further includes: a first update module, configured to update the trace file set according to the test metrics or the consumption of the graphics buffer; a third acquisition module, configured to acquire the updated sliding operation information and the updated consumption of the graphics buffer corresponding to the test cases from the updated trace file set; and a second update module, configured to update the test metrics according to the updated sliding operation information and the updated consumption of the graphics buffer.

[0151] In one embodiment, the first update module is specifically used to perform any of the following: determine the average number of graphics buffer consumptions and update the tracking file set based on the average number; update the tracking file set according to the maximum number of consecutive dropped frames; update the tracking file set according to frame rate information.

[0152] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0153] Based on the same inventive concept, this application also provides a chip, including a processor and a communication interface; the communication interface is used to receive or send data; the processor is configured to cause the chip to perform the steps in the above method embodiments.

[0154] It is understood that the chip involved in the embodiments of this application may be a field-programmable gate array (FPGA), may be an application-specific integrated circuit (ASIC), may be a system on chip (SoC), may be a central processor unit (CPU), may be a network processor (NP), may be a digital signal processor (DSP), may be a microcontroller unit (MCU), may be a programmable logic device (PLD), or other integrated chips, etc.

[0155] Based on the same inventive concept, this application also provides a chip module, such as... Figure 10 As shown, the chip module includes a communication module, a power module, a storage module, and a chip. Among them:

[0156] The power module is used to provide power to the chip module; the storage module is used to store data and instructions; the communication module is used for internal communication within the chip module, or for communication between the chip module and external devices; this chip corresponds to the chip in the above chip embodiment.

[0157] The implementation method of this chip module can be found in the relevant content of the above chip embodiment, and will not be repeated here.

[0158] In one exemplary embodiment, a terminal device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0159] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0160] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0161] It should be noted that the data (including but not limited to data used for analysis, storage and display) and information (including but not limited to information used for analysis, information stored, information displayed) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0162] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0163] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for testing an application, characterized in that, The method includes: Use a trajectory tracing tool to obtain a set of tracing files that display information during the execution of test cases in the target application; Obtain the sliding operation information and the amount of graphics buffer consumed corresponding to the test case from the tracking file set; Based on the sliding operation information and the amount of graphics buffer consumed, a test metric for the smoothness of the target application's operation is determined. The smoothness of the target application's operation is determined based on the test metrics.

2. The method according to claim 1, characterized in that, The sliding operation information includes the sliding time; the step of determining the test metrics for the smoothness of the target application's operation based on the sliding operation information and the consumption of the graphics buffer includes: Based on the sliding time and the amount consumed by the graphics buffer, the frame rate information during the execution of the target application is determined; Based on the amount consumed by the graphics buffer, determine the number of frame drops and the maximum number of consecutive frame drops during the execution of the target application; The frame rate information, the number of frame drops, and the maximum number of consecutive frame drops are determined as the test indicators.

3. The method according to claim 2, characterized in that, The step of determining the number of frame drops and the maximum number of consecutive frame drops during the execution of the target application based on the consumption of the graphics buffer includes: Based on the amount consumed by the graphics buffer, determine the valid frame drop events and continuous frame drop events that occur during the execution of the target application; The number of frame drops is determined based on the number of valid frame drop events; The maximum number of consecutive frame drops is determined from the consecutive frame drop events.

4. The method according to claim 3, characterized in that, The step of determining valid frame drop events and consecutive frame drop events occurring during the execution of the target application based on the consumption of the graphics buffer includes: Based on the amount consumed by the graphics buffer, determine the frame drop events and the continuous frame drop events that occur during the execution of the target application; A series of consecutive frame loss events are identified as the first valid frame loss event; All first and second frame drop events that occur during the execution of the target application are identified as valid frame drop events; the second frame drop event is any frame drop event other than the first frame drop event.

5. The method according to any one of claims 1-4, characterized in that, The step of obtaining the sliding operation information and the consumption of the graphics buffer corresponding to the test case from the tracking file set includes: Identify the abnormal tracking files in the tracking file set; The abnormal tracking files are removed to obtain a set of tracking files after removal; The sliding operation information and the consumption amount of the graphics buffer are obtained from the removed tracking file set.

6. The method according to claim 5, characterized in that, The process of determining the abnormal tracking files in the tracking file set includes: Obtain a first image of the interface displayed before the target application runs each operation in the test case, and a second image of the interface displayed after each operation is run; Determine the target operation corresponding to the difference between the first image and the second image being less than a preset threshold; The tracking file corresponding to the target operation is identified as the anomaly tracking file.

7. The method according to claim 2, characterized in that, The method further includes: Update the tracking file set based on the test metrics or the consumption of the graphics buffer; Obtain the updated sliding operation information and the updated consumption amount of the graphics buffer corresponding to the test case from the updated trace file set; The test metrics are updated based on the updated sliding operation information and the updated consumption of the graphics buffer.

8. The method according to claim 7, characterized in that, The step of updating the tracking file set based on the test metric or the consumption of the graphics buffer includes any one of the following: Determine the average number of times the graphics buffer is consumed, and update the tracking file set based on the average number of times the buffer is consumed; The tracking file set is updated based on the maximum number of consecutive lost frames; The tracking file set is updated based on the frame rate information.

9. A testing apparatus for an application, characterized in that, The device includes: The first acquisition module is used to acquire the set of trace files that display information during the execution of test cases in the target application through a trajectory tracing tool; The second acquisition module is used to acquire the sliding operation information and the consumption amount of the graphics buffer corresponding to the test case from the tracking file set; The first determining module is used to determine the test index of the running smoothness of the target application based on the sliding operation information and the consumption of the graphics buffer. The second determining module is used to determine the running smoothness of the target application based on the test indicators.

10. A chip, characterized in that, The device includes a processor and a communication interface, the processor being configured to cause the chip to perform the steps of the method described in any one of claims 1 to 8.

Images