Software testing method and device, electronic equipment and storage medium

Abstract

The invention provides a software testing method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a to-be-tested process; target threads associated with the to-be-tested process are determined, performance data of each target thread is recorded, and the target threads comprise threads with preset configurable attributes corresponding to the to-be-tested process and a preset number of threads with the highest resource occupancy rate on at least one resource; and obtaining a test result of the to-be-tested process according to the running log related to the to-be-tested process and the recorded performance data of each target thread at different moments.

Description

Technical Field

[0001] The present disclosure relates to the technical field of software testing, and in particular to a software testing method, device, electronic device and storage medium. Background Art

[0002] During the software testing process, in addition to checking whether there are logical problems in the software at the code level, and checking whether there are errors in the software and whether the software business can be implemented normally at the operation level, it is usually necessary to consider the performance issues of the software.

[0003] In related technologies, software performance is often measured only by some general performance parameters at the application or process level, such as the overall resource usage of the process. However, a process usually corresponds to the overall business of the application, and the resource usage of the process can only be used to judge the overall optimization degree of the application, but cannot provide the basis for judging specific problems to testers. Therefore, it is difficult for testers to know which specific aspects should be optimized and improved, which ultimately leads to poor software testing results. Summary of the invention

[0004] To overcome the problems existing in the related art, the present disclosure provides a software testing method, device, electronic device and storage medium.

[0005] A first aspect of the present disclosure provides a software testing method, the method comprising:

[0006] Get the process to be tested;

[0007] Determine target threads associated with the process to be tested, and record performance data of each target thread, wherein the target threads include threads having preset configurable attributes corresponding to the process to be tested, and a preset number of threads having the highest resource occupancy rate on at least one resource;

[0008] The test result of the process to be tested is obtained according to the running log related to the process to be tested and the performance data of each target thread recorded at different times.

[0009] Optionally, the method further includes:

[0010] When a log recording opportunity is reached, a preset log description corresponding to the log recording opportunity is recorded, and the operation log related to the process to be tested includes the recorded log description, and the log recording opportunity includes at least one of the following:

[0011] A logging instruction for triggering a preset operation related to the process to be tested is detected, and a preset code line is executed.

[0012] Optionally, obtaining the test result of the process to be tested according to the running log related to the process to be tested and the performance data of each target thread recorded at different times includes:

[0013] Aligning the time axis of the operation log related to the process to be tested and the performance data of each thread recorded, wherein the log and the performance data of the thread both have corresponding timestamps;

[0014] The test result of the process to be tested is obtained according to the aligned logs with the same time axis and the performance data of each target thread recorded at different times.

[0015] Optionally, the target thread is determined at at least one of the following information collection moments:

[0016] The time interval between the current moment and the last time when information collection is reached reaches the preset time length;

[0017] An information collection instruction for triggering a preset operation related to the process to be tested is detected;

[0018] The preset line of code is executed.

[0019] Optionally, the target thread also includes a thread whose resource occupancy rate on at least one resource is greater than an occupancy rate threshold corresponding to the at least one resource.

[0020] Optionally, the method further includes:

[0021] Determine the abnormality of the process to be tested according to the running log related to the process to be tested and at least one of the following information of the thread represented by the test result:

[0022] The time when the thread is created, the time when the thread is destroyed, the duration of the thread, and the resource usage of the thread.

[0023] Optionally, recording the performance data of each target thread includes:

[0024] Obtaining performance data of each thread to be tested, and determining whether the obtained performance data can be converted into a preset data type corresponding to the performance data;

[0025] Record performance data that can be converted into a preset data type corresponding to the performance data.

[0026] Optionally, the performance data includes at least one of the following:

[0027] The CPU occupancy rate of the thread, the current total CPU occupancy rate, the main memory occupancy rate of the thread, the current total main memory occupancy rate;

[0028] The recording of performance data of each target thread includes:

[0029] Based on at least one command line tool among the top tool, the dumpsys tool, and the cat tool, the performance data of each target thread is obtained and the performance data of each target thread is recorded.

[0030] A second aspect of the present disclosure provides a software testing device, the device comprising:

[0031] Process acquisition module, used to obtain the process to be tested;

[0032] A thread determination module, configured to determine target threads associated with the process to be tested, and record performance data of each target thread, wherein the target threads include threads having preset configurable attributes corresponding to the process to be tested, and a preset number of threads having the highest resource occupancy rate on at least one resource;

[0033] The result determination module is used to obtain the test result of the process to be tested according to the running log related to the process to be tested and the performance data of each target thread recorded at different times.

[0034] Optionally, the device further comprises:

[0035] A logging module is used to record a preset log description corresponding to the logging opportunity when the logging opportunity is reached, the operation log related to the process to be tested includes the recorded log description, and the logging opportunity includes at least one of the following: detecting a logging instruction for triggering a preset operation related to the process to be tested, and a preset code line being executed.

[0036] Optionally, the result determination module is used to obtain the test result of the process to be tested according to the running log related to the process to be tested and the performance data of each target thread recorded at different times, specifically for:

[0037] Aligning the time axis of the operation log related to the process to be tested and the performance data of each thread recorded, wherein the log and the performance data of the thread both have corresponding timestamps;

[0038] The test result of the process to be tested is obtained according to the aligned logs with the same time axis and the performance data of each target thread recorded at different times.

[0039] Optionally, the target thread is determined at at least one of the following information collection moments:

[0040] The time interval between the current moment and the last time when information collection is reached reaches the preset time length;

[0041] An information collection instruction for triggering a preset operation related to the process to be tested is detected;

[0042] The preset line of code is executed.

[0043] Optionally, the target thread also includes a thread whose resource occupancy rate on at least one resource is greater than an occupancy rate threshold corresponding to the at least one resource.

[0044] Optionally, the device further comprises:

[0045] The abnormality determination module is used to determine the abnormality of the process to be tested according to the operation log related to the process to be tested and at least one of the following information of the thread represented by the test result:

[0046] The time when the thread is created, the time when the thread is destroyed, the duration of the thread, and the resource usage of the thread.

[0047] Optionally, when the thread determination module is used to record the performance data of each target thread, it is specifically used to:

[0048] Obtaining performance data of each thread to be tested, and determining whether the obtained performance data can be converted into a preset data type corresponding to the performance data;

[0049] Record performance data that can be converted into a preset data type corresponding to the performance data.

[0050] Optionally, the performance data includes at least one of the following:

[0051] The CPU occupancy rate of the thread, the current total CPU occupancy rate, the main memory occupancy rate of the thread, the current total main memory occupancy rate;

[0052] When the thread determination module is used to record the performance data of each target thread, it is specifically used to:

[0053] Based on at least one command line tool among the top tool, the dumpsys tool, and the cat tool, the performance data of each target thread is obtained and the performance data of each target thread is recorded.

[0054] A third aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0055] A fourth aspect of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0056] A fifth aspect of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in the first aspect.

[0057] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0058] Based on the above implementation, the information collection granularity of software testing is refined to the thread level, so that the test results can more intuitively reflect the actual execution process of the software in the system, and then accurately reflect the performance bottleneck, so that the subsequent optimization process has a clearer goal. This technical effect is even more important in the context of the current widespread application of multi-threaded programming. In addition, due to the large number of threads and the thread numbers are often randomly assigned by the thread pool, the target thread related to the process to be tested is determined from the two aspects of the configurable properties of the thread and the resource occupation of the thread. This can further ensure the availability of the method and avoid too many threads that are not related to the process to be tested from being taken into consideration, so as to further improve the operator's testing experience and the final effect of software testing.

[0059] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0061] Figure 1 The following is a flowchart of a software testing method shown in some exemplary embodiments.

[0062] Figure 2 The following is a flowchart of a time axis alignment method shown in some exemplary embodiments.

[0063] Figure 3 is a schematic diagram of a test result shown by some exemplary embodiments.

[0064] Figure 4 The invention is a selection branch diagram of a software testing method shown in some exemplary embodiments.

[0065] Figure 5 is a block diagram of a software testing device showing some exemplary embodiments.

[0066] Figure 6 It is a hardware structure diagram of an electronic device shown in some exemplary embodiments. DETAILED DESCRIPTION

[0067] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0068] As described in the background technology, in the related art, software performance is often measured only by some generalized performance parameters (such as the overall memory usage of software applications) at the application software or process level, and these parameters cannot provide testers with good performance judgment conditions. For example, when testers find that the current memory usage of the software is too high, they can only re-examine the code from a macro perspective to complete code optimization. The operators have no sense of purpose, which is not convenient for developers to optimize specific codes and may cause some more fragmented code problems to be unable to be discovered in time. For example, there may be a logical deadlock in the code, which does not affect the overall operation of the software, but the accumulation of such errors will invisibly increase the operating overhead of the software, and these problems are difficult to determine through macro performance data, making the software testing effect poor.

[0069] In view of this, the present disclosure provides a software testing method, device, electronic device and storage medium. Next, the embodiments of the present disclosure are described in detail.

[0070] The first aspect of the present disclosure provides a software testing method. Figure 1 , which may include steps S101 to S103.

[0071] Step S101, obtaining the process to be tested.

[0072] For example, the process to be tested can be obtained based on any one or more of the following methods:

[0073] Reading a file storing at least one process to be tested (the file may be used to store the name of the process to be tested or other information that may be used to identify the process, and similar concepts in the following text apply) to obtain the process to be tested stored therein;

[0074] Reading a file storing at least one software application to be tested, and obtaining a process to which the software application to be tested belongs during the running of the software application to be tested;

[0075] Obtain a preset number of processes with the highest resource usage rate on at least one resource among currently running processes;

[0076] Obtain a process whose resource occupancy rate on at least one resource is greater than an occupancy rate threshold value corresponding to the at least one resource (in order to distinguish it from other occupancy rate threshold values ​​appearing later, the occupancy rate threshold value here may be referred to as a process occupancy rate threshold value, and the threshold value corresponding to a thread may be referred to as a thread occupancy rate threshold value later) in a currently running process;

[0077] Get all running threads.

[0078] It should be understood that the execution of the software application depends on the process. There may be one or more processes to be tested obtained in step S101. When multiple processes to be tested are obtained, steps S102, S103 and details related to these steps may be executed separately for each process to be tested, and different processes to be tested will not be distinguished in the following text.

[0079] Step S102, determining target threads associated with the process to be tested, and recording performance data of each target thread, wherein the target threads include threads having preset configurable attributes corresponding to the process to be tested, and a preset number of threads with the highest resource occupancy rate on at least one resource.

[0080] Since threads usually have the characteristics of being available on demand and destroyed after use, and are usually allocated by a thread pool or the system, that is to say, logically speaking, threads usually exist for functions (that is, for executing a certain piece of code), and their identification (that is, ID, Identity Document) is random.

[0081] Therefore, on the one hand, the target thread can be obtained based on the configurable properties of the thread, that is, the correspondence between the configurable properties (usually including keys and values) and the process to be tested (of course, it can also be a code block in the process, in which case the thread can have a known association with the specific code block) can be preset, and when the code block configures these configurable properties for the thread, in step S102, the thread related to (corresponding to) the process to be tested among the currently existing threads can be known.

[0082] For example, when writing software code, the configurable attributes of the called thread can be assigned values ​​at the place where the thread is called inside the function or business that needs to be tested. The configurable attributes of the thread may be related to the programming language, system version, and thread pool type; more commonly, the name attribute (name) and priority attribute (priority) of the thread can be assigned values ​​when the thread is instantiated to distinguish the code block to which the thread belongs or the process to which the thread belongs. For example, in a multi-threaded function existing in the process to be tested, the called thread is uniformly named "abc" (of course, other clear rules can also be followed here, such as the name attribute of the thread under a certain business uniformly starts with "abc"), then in the execution process of step S102, the currently existing thread with the name attribute of "abc" can be determined as the target thread, and at the same time, the corresponding relationship between the thread and the process (or a certain business, a certain code block) can also be determined at the same time. Based on the elimination method, the process or code block corresponding to other threads may also be determined.

[0083] On the other hand, the threads related to the process to be tested can be determined based on the resource occupancy of the threads. Exemplarily, the preset number of threads with the highest resource occupancy rate on at least one resource can be determined as the target threads (the preset number here can be specified by the operator, or can be determined in real time based on parameters such as the number of currently running processes or the number of threads before the execution of this step, such as the "preset number" increases by 2 for each thread); for example, the N1 threads with the highest CPU (Central Processing Unit) occupancy rate, the N2 threads with the highest memory (Main Memory) occupancy rate, or the N3 threads with the highest comprehensive resource occupancy rate calculated based on a preset algorithm and according to one or some performance parameters can all be determined as target threads (N1, N2, N3 are all preset positive integers). Exemplarily, the target threads also include threads whose resource occupancy rate on at least one resource is greater than the occupancy rate threshold corresponding to the at least one resource; that is, thresholds can be set for one or some resource occupancy rates (including comprehensive resource occupancy rates similar to those described above), and threads that meet these thresholds (for example, greater than these thresholds) are determined as target threads. This is because, in a software testing environment, threads with high consumption, high operating pressure, and high resource usage often require special attention, and they are likely to be related to a certain execution process of the process to be tested.

[0084] Since the number of threads is generally much larger than the number of processes, and developers usually do not write code to name threads (and configure other configurable properties) in every place where threads are used, we use static (determined based on the configurable properties of threads, which can be understood as the threads named by developers) and dynamic (taking the n threads with the highest resource usage) methods to comprehensively determine our target thread set, ensuring that the number of threads is neither too many nor too few, while ensuring their relevance to the process to be tested.

[0085] It should be understood that the resource occupancy rate mentioned above is a logical concept, which represents the degree of occupation of a certain (or some) system resources or hardware resources by a thread. According to the actual situation, it may not be a ratio (the meaning of "ratio" is that the memory occupancy rate is "1.2%), but an absolute value. For example, if 46MB of memory is occupied, the performance meaning represented by it may not change. In addition, taking the memory occupancy rate as an example, the resource occupancy rate of thread a can be relative to the total memory of the device (that is, the memory occupancy rate is the ratio of the amount of memory occupied by thread a to the total memory), or it can be relative to the currently occupied memory (that is, the memory occupancy rate is the ratio of the amount of memory occupied by thread a to the total memory currently occupied). The concepts of other "rates" in the present disclosure may also be applied to the above principles, and the present disclosure does not limit this.

[0086] In some embodiments, step S102 may have a preset trigger condition. For example, the target thread may be determined at at least one of the following information collection moments 1 to 3 (these information collection moments may be regarded as the triggering moment of step S102 as a whole):

[0087] Information collection time 1: The time interval between the current time and the last information collection opportunity reaches a preset time length, that is, the current target threads are periodically determined and the performance data of these target threads are recorded.

[0088] Information collection moment 2: An information collection instruction for triggering a preset operation related to the process to be tested is detected. For example, the method can detect instructions received by the process to be tested, and when a specific instruction is detected, or an instruction for triggering a line of code, a code block, or a functional module of the process to be tested, or a business (for the business and user levels, the above-mentioned triggered content can be regarded as executing a preset operation), the target thread is determined and the performance data of these target threads is recorded. For another example, the operator can issue these information collection instructions by operating preset controls (such as buttons), and each preset control can be used to indicate different preset operations, thereby achieving more targeted software testing.

[0089] Information collection moment 3: The preset code line is executed, that is, when a line of code or a code block is executed or a functional module or a business of the process to be tested is triggered.

[0090] Through the above triggering moments, the thread data recorded in step S102 will be able to have a stronger correlation with the real user operation or a certain logical execution process of the software, thereby improving the software testing effect. This is because the above moments can ensure that the performance data of the target thread is recorded in time in certain operations performed during the software testing process (of course, the target thread targeted by each recording process can be dynamically acquired, that is, the target thread is first determined, and then the performance data is recorded for each target thread), so that the recorded performance data can more accurately reflect the issues that the operator is concerned about, avoiding the problem of low correlation between the recorded performance data and the execution process of the software.

[0091] Performance data refers to various performance-related indicators (i.e., parameters) that are used to characterize the system, application, process, thread, or device during operation. Common examples include: CPU occupancy of the thread (the percentage of CPU used by the thread at a specific time or time period), the current total CPU occupancy, the main memory occupancy of the thread (the percentage of memory capacity used by the thread at a specific time or time period), the current total main memory occupancy, the running time of the thread (the total time consumed by the thread from startup to the current time or being destroyed), I / O operation parameters (I / O refers to INPUT / OUTPUT, which means input / output. This parameter can be used to characterize the number of input / output operations performed by the thread and the time consumed), the number of context switches (context can refer to the information required for thread operation, such as program counters, registers, memory addresses, etc. This parameter can characterize the frequency of thread switching between different states), etc. These parameters can more accurately reflect the possible problems and areas to be optimized in the thread to be tested and even the application software through thread-level data, thereby improving the software testing effect.

[0092] Adaptively, taking Linux and its derivative systems as an example, the recording of the performance data of each target thread may include: obtaining the performance data of each target thread based on at least one command line tool among the top tool, the dumpsys tool, and the cat tool, and recording the performance data of each target thread.

[0093] When using the top command line tool, you can use "adb shell top -Hn 1 | grep<thread_id> " and similar commands to determine the CPU usage of a specific thread. When using the dumpsys command-line tool, you can use "adb shelldumpsys meminfo<package_name> " and similar commands to determine memory usage of a specific package (<package_name> When using the cat command-line tool, you can determine the total system memory usage (i.e. total occupancy) based on "adb shellcat / proc / meminfo" and similar commands.

[0094] During or after the process of obtaining the performance data of the thread, the obtained performance data may also be subjected to a legality check to avoid recording data that does not conform to the rules, which may cause errors in the subsequent execution process or occupy too much useless space. Specifically, recording the performance data of each target thread may include: obtaining the performance data of each thread to be tested, and determining whether the obtained performance data can be converted into a preset data type corresponding to the performance data; and recording the performance data that can be converted into the preset data type corresponding to the performance data.

[0095] Specifically, the acquisition of performance data may be affected by complex uncertain factors such as code logic vulnerabilities, operations at multiple levels running in the system, and vulnerabilities in the system, resulting in illegal data being recorded in the end. For example, the return value obtained by obtaining the CPU usage of a thread based on the top command line tool may be a string type of data. In this case, you can first adaptively process the string to cut out the specific part containing the CPU usage, and then try to convert this part of the data (currently still a string type) to a floating point type (for example, if you want to record the CPU usage as a float type, you can try to convert the string type to a float type). If the conversion is successful, you can record the value; if the conversion fails, you can discard the data or try to re-acquire it and repeat the above process again. This process improves the robustness of the method.

[0096] It should be understood that the "recording" process involved in the present disclosure is a logical concept, which can be stored in certain data structures or stored in key-value storage databases (such as some memory databases such as redis), or can be persistently stored in external memory. These "recorded" data are only used to distinguish from "unrecorded" data, and these "recorded" data can be obtained in step S103. During or after the recording process, the data to be recorded (or being recorded) can be matched with the historical data that has been recorded, and the data to be recorded (or being recorded) can be adaptively matched and stored with the matched historical data. For example, the data structure for recording the CPU occupancy of a thread can be in the form of an array. When the current CPU occupancy of thread a is about to be recorded, it is found that there is historical data of the CPU occupancy of thread a, then the current CPU occupancy of thread a can be directly recorded in the data structure (or database entry) corresponding to the historical data to realize the data sorting process at the thread level.

[0097] Step S103, obtaining a test result of the process to be tested according to the running log related to the process to be tested and the performance data of each target thread recorded at different times.

[0098] Run logs record various information generated by a process during execution. Generally speaking, the operating system will record system event logs when a process starts, terminates, or context switches; applications will generate error logs when errors or exceptions occur (even if the log code is not manually written); third-party libraries and third-party components may record debugging information and performance data for monitoring and debugging. These logs can be used to reflect the specific execution process that the process under test goes through during the test, such as what operations were performed at what time, and then when it crashed. After obtaining this information, combined with the performance data of each target thread recorded at different times, the specific reasons for optimization problems and application crashes can be analyzed in detail (even accurate to the code block and function level).

[0099] In addition to the above-mentioned automatically generated logs that can be used as running logs related to the process to be tested, for specific test scenarios and preset test processes, the method may also include: when the log recording timing is reached, recording a preset log description corresponding to the log recording timing, the running log related to the process to be tested includes the recorded log description, and the log recording timing includes at least one of the following: detecting a log recording instruction for triggering a preset operation related to the process to be tested, and a preset code line is executed.

[0100] Among them, the details of "triggering the preset operation related to the process to be tested" and "preset code line" can refer to the introduction of information collection time 2 and information collection time 3 in the previous text. In other words, after reaching these log recording opportunities, the log descriptions corresponding to these opportunities can be deliberately recorded, and the recorded log descriptions can be used as the running logs related to the process to be tested. For example, if a button is preset to trigger function A in the process to be tested, then optionally, the button can have the following functions in dimensions: presenting to the user and receiving instructions from the user, triggering function A, triggering information collection instructions, and triggering log recording instructions. The correspondence between the log recording opportunity and the log description can be specified by the operator. For example, the log description corresponding to the above button is set to "Function A starts to execute"; and the last line or lines of code in function A can be used to trigger another log recording opportunity, and the log description corresponding to this opportunity can be "Function A has been executed." Based on this, the correlation between the running log and the process to be tested can be further ensured, making the final test results more intuitive and improving the software testing effect.

[0101] In some embodiments, obtaining the test result of the process to be tested based on the running logs related to the process to be tested and the performance data of each target thread recorded at different times may include: aligning the time axis of the running logs related to the process to be tested and the performance data of each thread recorded, wherein the logs and the performance data of the threads have corresponding timestamps; and obtaining the test result of the process to be tested based on the aligned logs having the same time axis and the performance data of each target thread recorded at different times.

[0102] Specifically, since the two processes of log recording and performance data recording can be asynchronous, can have different trigger conditions, and are based on time (or time period) (that is, the operation log and performance data are actually "operation logs at a certain moment" and "performance data at a certain moment"), in other words, the recorded performance data and log descriptions may actually be discrete data, and the two constitute their own time axes (because their time axes are actually discrete and may not overlap with each other). See Figure 2 , which takes the software testing process that needs to test two functions, function A and function B, as an example, and shows an optional timeline alignment method.

[0103] The first timeline is the timeline of the running logs, and the information represented by each log is: at time t11, function A starts to execute; at time t12, function B starts to execute; at time t13, function A is completed; at time t14, function B is completed. The second timeline is the timeline of the performance data. In this example, the performance data is recorded periodically. At times t21, t22, t23, and t24, four performance data items, record 1, record 2, record 3, and record 4, are recorded in sequence.

[0104] After the timeline is aligned, it can be known that record 1 is recorded when function A is executing and function B has not started to execute, record 2 and record 3 are recorded when both function A and B are executing, and record 4 is recorded when function A has completed execution and function B is in the process of execution. Then, this timeline can be sorted out to obtain the corresponding relationship between the performance data of the thread and the running log ("alignment" can be understood as the process of determining the corresponding relationship between the two parallel data, the running log and the performance data). In an example, the aligned running log and performance data can use the performance data of the thread as the timeline, that is, the running log and performance data are integrated into a number of data consisting of parameters of the following dimensions: timestamp, scenario description, process ID, thread ID, thread CPU usage, thread total CPU usage, thread memory usage, thread total memory usage; wherein, the subject corresponding to each parameter can be "performance data", that is, at what moment the performance data is, what is the scenario description at that moment (the scenario can be determined based on the running log, for example, the currently executed operation is "login operation"), what is the thread ID corresponding to the performance data, what is the process to which it belongs, and so on.

[0105] It is worth noting that arranging and presenting the operation logs and performance data directly in sequence according to a unified timeline (ie, there is no need to determine the direct correspondence between the operation logs and the performance data, but directly displaying these discrete data on the timeline) is also an optional implementation.

[0106] The final test results can be presented to the user based on text, various charts or higher-dimensional presentation methods (can also be stored for subsequent use), and used to characterize part or all of the contents of the running log related to the process to be tested and part or all of the performance data recorded for each target thread at different times, or data of other dimensions obtained based on these contents.

[0107] After the timeline is aligned, the correspondence between performance data and operation logs has been determined. Then, an attempt can be made to judge the abnormal situation based on the preset judgment conditions, and the judgment result can be presented to the user.

[0108] Then, in some embodiments, the method may further include:

[0109] The abnormality of the process to be tested is determined according to the operation log related to the process to be tested and at least one of the following information of the thread represented by the test result: the creation time of the thread, the destruction time of the thread, the duration of the thread, and the resource occupancy rate of the thread.

[0110] The form of the test results may be related to the preset algorithm and the specific interaction capabilities of the device, which is not limited in this disclosure. Figure 3 , which shows a schematic diagram of the test results in the form of a chart. Among them, based on the log information and the configurable properties of the thread, it has been determined that thread a is only related to function A, thread b is only related to function B, and thread c is only related to function A (the specific determination method can be comprehensively referred to the previous content).

[0111] According to the development process of function A and function B, we can know the approximate operating pressure of these functions. Then, we can set judgment conditions based on the above-mentioned parameter dimensions of the threads related to them, and compare the data of each thread with the judgment conditions during the execution of the method to determine whether the corresponding process has problems. For example, we can determine the preset reasonable range of the parameters corresponding to the performance data (for example, the destruction time of the thread created by function A should not be later than the execution completion time of function A), and determine whether the performance data is abnormal based on the preset reasonable range. Next, we combine Figure 3 , examples of the above dimensions are introduced.

[0112] For example, thread a is only related to function a, but the performance data of thread a is still collected at time t24 (when function A has been executed). Then, based on the exception of thread a at the "thread destruction moment", an exception judgment of "function A may not release resources in time" can be given, and corresponding suggestions can be presented to the user.

[0113] For another example, thread b is only related to function B, but function B only calls the thread in a tiny step in its processing (the execution pressure of this step is very small). That is to say, the CPU usage of the thread related to function B should not exceed a certain threshold, and the duration of the thread related to function B should not be too long. Figure 3 In the example shown, thread b exists in all performance data collected after function B starts executing, and its CPU usage is high. In fact, there may be optimizable issues such as deadlock and code redundancy in function B. During the execution of the method, the anomaly of "function B may have deadlock" can be judged from the dimensions of "thread duration" and "thread resource usage".

[0114] For another example, if function A uses multithreading technology only in the first step of its code, then the time interval between the creation time of the thread related to function A and the start time of function A's execution may not be very large, but in Figure 3 In the example shown, the time interval between time t21 and the execution start time t11 of function A is already very large, but the method does not record the performance data of thread c related to function A at time t21. This may be because thread c was instantiated too late. The method can determine the anomaly of "there is a large delay in thread creation by function A" related to the process under test based on the dimension of "thread creation time".

[0115] Based on the above implementation mode, the information collection granularity of software testing is refined to the thread level; and for software applications, when executing code blocks, they usually need to be executed through threads. Refining the granularity of information collection to the thread level can more intuitively reflect the actual execution process of the software in the system, so that the software test results can correspond to a certain code block or a certain operation (rather than corresponding to the entire process or the entire application), thereby significantly improving the amount of information contained in the software test results, making it easier for operators to specifically identify unreasonable performance usage and difficult-to-identify code logic problems (such as code deadlock, unreleased resources) during software operation, and can more accurately reflect operating bottlenecks, so that the subsequent optimization process has clearer goals and greatly improves the software testing effect. This technical effect is even more important in the context of the current widespread application of multi-threaded programming. In addition, since thread numbers are often randomly assigned by a thread pool and the number of threads is generally much larger than the number of processes, determining the process of target threads related to the process to be tested (the process to be tested can also be understood as the application to be tested, because the relationship between the application and the process is usually known) from two aspects: the configurable properties of the threads and the resource occupation of the threads can further ensure the availability of the method and avoid taking too many threads that are not related to the process to be tested into consideration, so as to further improve the operator's testing experience and the final effect of software testing.

[0116] Next, combine Figure 4 The selection branch diagram shown comprehensively explains the method shown in the first aspect.

[0117] First, in step S401, the method obtains a process file to be tested, or determines a process to be tested according to an application to be tested corresponding to at least one process to be tested and the application. Then, at the information collection moment, the following steps are further performed for each process to be tested.

[0118] In step S402a, the method obtains a preset number of threads with the highest resource occupancy rate on at least one resource, and in step S402b, based on preset information related to the process to be tested, obtains threads having configurable information corresponding to the process to be tested, and the threads obtained in steps S402a and S402b constitute (one or more) target threads.

[0119] Next, for each target thread, in step S404a, the CPU performance data related to the target thread is collected based on the top tool, and in step S404b, the memory data related to the target thread is collected based on the dumpsys tool and the cat tool. You can first try to convert these collected data into corresponding preset data types (S405), record the successfully converted data (S406), and the recorded data are summarized to form the performance data of the target thread.

[0120] In addition, when the log recording opportunity is reached, the method will also record the log recording description corresponding to the log recording opportunity (S411). These log descriptions, as well as optional operation logs recorded by other systems, application software, and third-party components, are used together to form the operation log related to the process to be tested (S412).

[0121] In the result output stage, in step S407, the recorded performance data and the operation log related to the process to be tested can be aligned in time to determine the scenario description corresponding to each performance data (determined based on the operation log). Next, in step S408, the data can be integrated, calculated (for example, the average of a certain performance data within a certain period of time, the total I / O volume of a certain thread, etc.), plotted, and displayed (for example, based on text, various charts, or higher-dimensional presentation methods).

[0122] Finally, based on the preset conditions, it can be determined in step S409 whether the integrated and calculated data (or original data) meets the abnormal condition. If so, the thread to be tested is determined to be abnormal and reported in step S4010. In addition, the integrated and calculated data (or original data), as well as the determined abnormal results, can be persistently stored (S4011) for subsequent data analysis.

[0123] Corresponding to the embodiments of the aforementioned method, the present disclosure also provides embodiments of a device and a terminal to which the device is applied.

[0124] The second aspect of the present disclosure provides a software testing device. Figure 5 , the device comprises:

[0125] The process acquisition module 501 is used to acquire the process to be tested;

[0126] A thread determination module 502, configured to determine target threads associated with the process to be measured, and record performance data of each target thread, wherein the target threads include threads having preset configurable attributes corresponding to the process to be measured, and a preset number of threads having the highest resource occupancy rate on at least one resource;

[0127] The result determination module 503 is used to obtain the test result of the process to be tested according to the running log related to the process to be tested and the performance data of each target thread recorded at different times.

[0128] Optionally, the device further comprises:

[0129] A logging module is used to record a preset log description corresponding to the logging opportunity when the logging opportunity is reached, the operation log related to the process to be tested includes the recorded log description, and the logging opportunity includes at least one of the following: detecting a logging instruction for triggering a preset operation related to the process to be tested, and a preset code line being executed.

[0130] Optionally, the result determination module 503 is used to obtain the test result of the process to be tested according to the running log related to the process to be tested and the performance data of each target thread recorded at different times, specifically for:

[0131] Aligning the time axis of the operation log related to the process to be tested and the performance data of each thread recorded, wherein the log and the performance data of the thread both have corresponding timestamps;

[0132] The test result of the process to be tested is obtained according to the aligned logs with the same time axis and the performance data of each target thread recorded at different times.

[0133] Optionally, the target thread is determined at at least one of the following information collection moments:

[0134] The time interval between the current moment and the last time when information collection is reached reaches the preset time length;

[0135] An information collection instruction for triggering a preset operation related to the process to be tested is detected;

[0136] The preset line of code is executed.

[0137] Optionally, the target thread also includes a thread whose resource occupancy rate on at least one resource is greater than an occupancy rate threshold corresponding to the at least one resource.

[0138] Optionally, the device further comprises:

[0139] The abnormality determination module is used to determine the abnormality of the process to be tested according to the operation log related to the process to be tested and at least one of the following information of the thread represented by the test result:

[0140] The time when the thread is created, the time when the thread is destroyed, the duration of the thread, and the resource usage of the thread.

[0141] Optionally, when the thread determination module 502 is used to record the performance data of each target thread, it is specifically used to:

[0142] Obtaining performance data of each thread to be tested, and determining whether the obtained performance data can be converted into a preset data type corresponding to the performance data;

[0143] Record performance data that can be converted into a preset data type corresponding to the performance data.

[0144] Optionally, the performance data includes at least one of the following:

[0145] The CPU occupancy rate of the thread, the current total CPU occupancy rate, the main memory occupancy rate of the thread, the current total main memory occupancy rate;

[0146] The thread determination module 502 is used to record the performance data of each target thread, specifically to:

[0147] Based on at least one command line tool among the top tool, the dumpsys tool, and the cat tool, the performance data of each target thread is obtained and the performance data of each target thread is recorded.

[0148] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0149] A third aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0150] For the device embodiments and computer program product embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. In addition, the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative labor.

[0151] In a fourth aspect, the software testing apparatus provided by the present disclosure can be applied to electronic devices. Figure 6 , which exemplarily shows a hardware schematic diagram of an electronic device. For example, the device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0152] The device 600 may include one or more of the following components: a processing component 601 , a memory 602 , a power component 603 , a multimedia component 604 , an audio component 605 , an input / output (I / O) interface 606 , a sensor component 607 , and a communication component 608 .

[0153] The processing component 601 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 601 may include one or more processors 609 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 601 may include one or more modules to facilitate the interaction between the processing component 601 and other components. For example, the processing component 601 may include a multimedia module to facilitate the interaction between the multimedia component 604 and the processing component 601.

[0154] The memory 602 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0155] The power component 603 provides power to the various components of the device 600. The power component 603 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 600.

[0156] The multimedia component 604 includes a screen providing an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 604 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0157] The audio component 605 is configured to output and / or input audio signals. For example, the audio component 605 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 602 or sent via the communication component 608. In some embodiments, the audio component 605 also includes a speaker for outputting audio signals.

[0158] I / O interface 606 provides an interface between processing component 601 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0159] The sensor assembly 607 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 607 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, and the sensor assembly 607 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600, and the temperature change of the device 600. The sensor assembly 607 can also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 607 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 607 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0160] The communication component 608 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G or 5G or a combination thereof. In an exemplary embodiment, the communication component 608 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 608 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0161] In an exemplary embodiment, the device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the software testing method of the above-mentioned electronic device.

[0162] In a fifth aspect, the present disclosure, in an exemplary embodiment, further provides a non-transitory computer-readable storage medium including instructions, such as a memory 602 including instructions, and the instructions can be executed by a processor 609 of a device 600 to complete the software testing method of the electronic device. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0163] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0164] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the inventions claimed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not claimed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0165] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0166] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A software testing method, characterized in that: The method comprises: Get the process to be tested; Determine target threads associated with the process to be tested, and record performance data of each target thread, wherein the target threads include threads having preset configurable attributes corresponding to the process to be tested, and a preset number of threads having the highest resource occupancy rate on at least one resource; The test result of the process to be tested is obtained according to the running log related to the process to be tested and the performance data of each target thread recorded at different times.

2. The software testing method according to claim 1, characterized in that: The method further comprises: When the logging opportunity is reached, a preset log description corresponding to the logging opportunity is recorded, and the running log related to the process to be tested includes the recorded log description. The logging opportunity includes at least one of the following: a logging instruction for triggering a preset operation related to the process to be tested is detected, and a preset code line is executed.

3. The software testing method according to claim 2, characterized in that: The step of obtaining the test result of the process to be tested according to the running log related to the process to be tested and the performance data of each target thread recorded at different times includes: Aligning the time axis of the operation log related to the process to be tested and the performance data of each thread recorded, wherein the log and the performance data of the thread both have corresponding timestamps; The test result of the process to be tested is obtained according to the aligned logs with the same time axis and the performance data of each target thread recorded at different times.

4. The software testing method according to claim 1, characterized in that: The target thread is determined at at least one of the following information collection moments: The time interval between the current moment and the last time when information collection is reached reaches the preset time length; An information collection instruction for triggering a preset operation related to the process to be tested is detected; The preset line of code is executed.

5. The software testing method according to claim 1, characterized in that: The target thread also includes a thread whose resource occupancy rate on at least one resource is greater than an occupancy rate threshold corresponding to the at least one resource.

6. The software testing method according to claim 1, characterized in that: The method further comprises: Determine the abnormality of the process to be tested according to the running log related to the process to be tested and at least one of the following information of the thread represented by the test result: The time when the thread is created, the time when the thread is destroyed, the duration of the thread, and the resource usage of the thread.

7. The software testing method according to claim 1, characterized in that: The recording of performance data of each target thread includes: Obtaining performance data of each thread to be tested, and determining whether the obtained performance data can be converted into a preset data type corresponding to the performance data; Record performance data that can be converted into a preset data type corresponding to the performance data.

8. The software testing method according to claim 1, characterized in that: The performance data includes at least one of the following: the CPU occupancy rate of the thread, the current total occupancy rate of the CPU, the main memory occupancy rate of the thread, and the current total occupancy rate of the main memory; The recording of performance data of each target thread includes: Based on at least one command line tool among the top tool, the dumpsys tool, and the cat tool, the performance data of each target thread is obtained and the performance data of each target thread is recorded.

9. A software testing device, characterized in that: The device comprises: Process acquisition module, used to obtain the process to be tested; A thread determination module, configured to determine target threads associated with the process to be tested, and record performance data of each target thread, wherein the target threads include threads having preset configurable attributes corresponding to the process to be tested, and a preset number of threads having the highest resource occupancy rate on at least one resource; The result determination module is used to obtain the test result of the process to be tested according to the running log related to the process to be tested and the performance data of each target thread recorded at different times.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images