A cluster monkey automatic test method and system

Abstract

The application relates to the technical field of data processing, in particular to a cluster type Monkey automatic test method and system. The method comprises the following steps: identifying a risk device and analyzing the initial priority, resource demand and conflict risk coefficient of each risk task in the risk device to determine the scheduling priority of each risk task in the risk device; scheduling each risk task in the risk device to the task cluster of the corresponding target device in sequence according to the scheduling priority of the risk task; performing weighted processing on the use rate of various resources by the device corresponding task cluster and the preset weight corresponding to each type of resource to obtain the comprehensive use rate of the basic resource quota by the task cluster; and optimizing the basic resource quota of the task cluster according to the comprehensive use rate of the basic resource quota by the task cluster to obtain the target resource quota of each device corresponding task cluster. The application can improve the resource utilization efficiency of the cluster type Monkey automatic test.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a cluster-based automated testing method and system for Monkey. Background Technology

[0002] Application software undergoes monkey testing during development or before release to determine if any stability issues exist. Clustered Monkey testing is a distributed upgrade of native Monkey, used to schedule a cluster of multiple heterogeneous devices through a central node to achieve large-scale, high-concurrency, and fully automated stress testing of app stability and compatibility.

[0003] In related technologies, the main approach is to assign fixed equipment to each task derived from the breakdown of the project under test. However, the actual execution time of different tasks during the testing process varies, resulting in some tasks being completed first, leaving the equipment idle for extended periods, while other tasks suffer delays due to insufficient resources.

[0004] This shows that the cluster-based Monkey automated testing method in related technologies has low resource utilization. Summary of the Invention

[0005] The purpose of this invention is to provide a clustered Monkey automated testing method and system to solve the technical problem of low resource utilization in clustered Monkey automated testing methods in related technologies.

[0006] In a first aspect, embodiments of the present invention provide a clustered Monkey automated testing method, the method comprising: Risk devices are identified among multiple devices, and the scheduling priority of each risk task in the risk device is determined by analyzing the initial priority, resource requirements and conflict risk coefficient of each risk task in the risk device. The risk device is a device whose task load value is greater than the load threshold, the risk task is an unprocessed task whose basic priority is higher than the priority threshold, and the conflict risk coefficient is used to characterize the maximum risk of resource conflict when the corresponding risk task runs with other tasks at the same time. According to the scheduling priority of each risk task in the risk device, each risk task in the risk device is scheduled to the task cluster of the corresponding target device in turn. The target device is other devices whose available resources are greater than or equal to the start resource threshold of the corresponding risk task. The scheduling order of risk tasks is positively correlated with the scheduling priority of the risk task. Based on the utilization rate of various resources by the task cluster corresponding to each device and the weight of each resource, the comprehensive utilization rate of the basic resource quota of the task cluster corresponding to each device is obtained. Based on the overall utilization rate of the basic resource quota for each task cluster corresponding to each device, the basic resource quota for each task cluster corresponding to each device is optimized to obtain the target resource quota for each task cluster corresponding to each device.

[0007] In one embodiment, determining the scheduling priority of each risk task in the risk device by analyzing its initial priority, resource requirements, and conflict risk coefficient includes: Based on preset correlation values, preset risk coefficients, and preset urgency coefficients, the basic priority of risk tasks is optimized to obtain the initial priority of risk tasks. The initial priority is proportional to the basic priority of the corresponding risk task, proportional to the preset correlation value of the corresponding risk task, proportional to the preset risk coefficient of the corresponding risk task, and proportional to the preset urgency coefficient of the corresponding risk task. Data analysis is performed based on the unit consumption baseline of the risk task, the number of events included in the risk task, and the preset risk coefficient to obtain the resource requirement of the risk task. The resource requirement of the risk task is proportional to the unit consumption baseline of the corresponding risk task, the number of events included in the corresponding risk task, and the preset risk coefficient of the corresponding risk task. The unit consumption baseline of the risk task is calculated based on the corresponding historical statistical values ​​and the basic resource quota of the project to be tested. Based on the initial priority of the risk task, the resource requirements of the risk task, the available resources of the equipment corresponding to the risk task, and the conflict risk coefficient of the risk task, data analysis is performed to obtain the scheduling priority of the risk task. The scheduling priority of the risk task is proportional to the initial priority of the risk task, and the scheduling priority of the risk task is proportional to the conflict risk coefficient of the risk task.

[0008] In one embodiment, optimizing the basic resource quota for each task cluster corresponding to each device based on the overall utilization rate of the basic resource quota for each task cluster corresponding to each device, to obtain the target resource quota for each task cluster corresponding to each device, includes: By analyzing the relative magnitudes of the resource redundancy, the first redundancy threshold, and the second redundancy threshold corresponding to the target task cluster, an adjustment coefficient for the resource quota of the target task cluster is obtained. The target task cluster is any one of multiple task clusters corresponding to multiple devices. The resource redundancy is used to characterize the proportion of unused resources. The first redundancy threshold is greater than the second redundancy threshold. The adjustment coefficient of the target task cluster is calculated based on the comprehensive utilization rate of the target task cluster to the corresponding target device and a preset adjustment step size. Calculate the adjustment coefficient of the resource quota of the target task cluster, and multiply it by the basic resource quota of the target task cluster to obtain the target resource quota of the target task cluster.

[0009] In one embodiment, the process of analyzing the relative magnitudes of the resource redundancy, the first redundancy threshold, and the second redundancy threshold corresponding to the target task cluster to obtain the adjustment coefficient of the resource quota for the target task cluster includes: When the resource redundancy corresponding to the target task cluster is greater than the first redundancy threshold, the adjustment coefficient of the resource quota of the target task cluster is the difference between the first product value and the preset coefficient, wherein the first product value is the product of the idle rate and the preset adjustment step size, and the idle rate is used to characterize the proportion of idle resources. If the resource redundancy corresponding to the target task cluster is greater than the second redundancy threshold and less than or equal to the first redundancy threshold, the adjustment coefficient of the resource quota of the target task cluster is directly assigned to the preset target coefficient. When the resource redundancy corresponding to the target task cluster is less than or equal to the second redundancy threshold, the adjustment coefficient of the resource quota of the target task cluster is the sum of the second product value and the preset coefficient, wherein the second product value is the product of the overload rate and the preset adjustment step size, and the overload rate is used to characterize the proportion of overloaded resources.

[0010] In one embodiment, before identifying the risky device among multiple devices, the method further includes: The throughput of the device cluster is obtained by performing feature analysis based on the processing rate of each device and the historical crash rate of each device. The device cluster is a cluster composed of all devices. The throughput of the device cluster is positively correlated with the processing rate of multiple devices and negatively correlated with the historical crash rate of multiple devices. The test item is split according to the event type to obtain multiple event streams. Each event stream corresponds to an event sequence of an event type, including at least one of touch screen events and click events. The event sequence is a sequence of multiple events of the same event type. For each event stream, feature analysis is performed based on the number of events included in each event stream, the processing rate of each device, the throughput of the device cluster, and the risk correction coefficient to obtain the number of events corresponding to each device for each event stream. Among them, the number of events corresponding to each event stream is proportional to the processing rate of the corresponding device, proportional to the number of events included in the corresponding event stream, proportional to the risk correction coefficient, and inversely proportional to the throughput of the device cluster. For each device, based on the number of events corresponding to each device in each event stream, the corresponding number of events are obtained from multiple event streams to obtain multiple events corresponding to each device; For each device, the multiple events corresponding to each device are split and processed according to the rated processing threshold of each device to obtain multiple tasks corresponding to each device. The rated processing threshold is used to characterize the maximum number of events that can be safely executed by the corresponding device in a single instance.

[0011] In one embodiment, the step of obtaining the processing rate of each device includes: Analyze the fluctuation of historical data for each type of resource to obtain the weight of each type of resource. The resource types include at least: CPU specifications, memory capacity, and remaining storage space. The target device's comprehensive performance index is obtained by weighting the attribute values ​​of multiple types of resources of the target device and the weight of each type of resource. The comprehensive performance index is used to characterize the processing capability of the corresponding device. The target device is any device in the device cluster. Based on the comprehensive performance index of the target device and the unit event consumption time of the target device in each event type, the ratio is calculated to obtain the type-specific processing rate of the target device in each event type. The target device's processing rate is obtained by weighting the processing rates of the target device under multiple event types and the proportion of each event type in the test items.

[0012] In one embodiment, the step of obtaining the rated processing threshold for each device includes: Based on the target device’s processing rate, preset execution time and risk correction coefficient, a product operation is performed to obtain the target device’s initial rated processing threshold. The initial rated processing threshold of the target device is rounded down to obtain the rated processing threshold of the target device.

[0013] In one embodiment, the step of obtaining the basic resource quota for each task cluster includes: The basic resource quota of the test project is obtained by merging the basic priority of the test project and the total resource volume of the device cluster. The basic resource quota of the test project is proportional to the basic priority of the test project and the basic resource quota of the test project is proportional to the total resource volume. Based on the proportion of the resource requirements of each task cluster to the resource requirements of the project under test, the basic resource quota of the project under test is allocated to each task cluster proportionally, thus obtaining the basic resource quota of each task cluster.

[0014] In one embodiment, the method further includes: Take a snapshot of each assigned task after the test is completed to obtain the status record of the completed task.

[0015] Secondly, another embodiment of the present invention provides a clustered Monkey automated testing system, the system comprising: The analysis module is used to identify risky devices among multiple devices and analyze them based on the initial priority, resource requirements and conflict risk coefficient of each risky task in the risky device to determine the scheduling priority of each risky task in the risky device. The risky device is a device whose task load value is greater than the load threshold, the risky task is an unprocessed task with a basic priority higher than the priority threshold, and the conflict risk coefficient is used to characterize the maximum risk of resource conflict when the corresponding risky task runs with other tasks at the same time. The scheduling module is used to schedule each risk task in the risk device to the task cluster of the corresponding target device in sequence according to the scheduling priority of each risk task in the risk device. The target device is other devices whose available resources are greater than or equal to the start resource threshold of the corresponding risk task. The scheduling order of the risk tasks is positively correlated with the scheduling priority of the risk task. The calculation module is used to perform weighted processing based on the utilization rate of various resources of each device's corresponding task cluster and the weight of each resource, to obtain the comprehensive utilization rate of the basic resource quota of each device's corresponding task cluster. The optimization module is used to optimize the basic resource quota of each task cluster corresponding to each device based on the overall utilization rate of the basic resource quota of each task cluster, so as to obtain the target resource quota of each task cluster corresponding to each device.

[0016] The present invention has the following beneficial effects: This invention identifies risky devices among multiple devices and analyzes the initial priority, resource requirements, and conflict risk coefficients of each risky task within the risky device to determine the scheduling priority of each risky task. Based on this priority, each risky task in the risky device is sequentially scheduled to the task cluster of the corresponding target device. Then, based on the utilization rate of various resources and their corresponding weights within each device's corresponding task cluster, a weighted average is calculated to obtain the comprehensive utilization rate of the basic resource quota for each device's corresponding task cluster. Finally, based on this comprehensive utilization rate, the basic resource quota for each device's corresponding task cluster is adjusted. Optimization yields the target resource quota for each task cluster corresponding to each device. On one hand, by first identifying risky devices and then scheduling and optimizing resources for risky tasks, the problem of device overload can be alleviated, and the task interruption caused by device downtime can be avoided by re-execution, thus improving the stability of testing. On the other hand, by prioritizing the scheduling of high-priority risky tasks to other devices, high-priority tasks can obtain resources in a timely manner and be executed first, thus improving the testing timeliness of high-priority tasks. Furthermore, by dynamically adjusting the resource quota of each task cluster based on the overall utilization rate, the resource allocation is matched with the real-time load, avoiding resource idle waste or overload contention, thereby improving the resource utilization efficiency of clustered Monkey automated testing. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a clustered Monkey automated testing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a clustered Monkey automated testing system provided in an embodiment of the present invention. Detailed Implementation

[0018] The following embodiments of the present invention can be applied to device clusters, such as control nodes within a device cluster. For example, they can be executed by a controller within the control node. Specifically, the controller may be equipped with a scheduler, and the following embodiments can be executed by this scheduler. The device can be a mobile device or a fixed device. For example, mobile devices include, but are not limited to, laptops, smartphones, and tablets. Fixed devices include, but are not limited to, computers and servers. The specific solution of a clustered Monkey automated testing method and system provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0019] This invention proposes a clustered automated Monkey testing method. Please refer to [link / reference]. Figure 1The diagram illustrates a flowchart of a clustered Monkey automated testing method according to an embodiment of the present invention, which includes the following steps: Step S1: Identify risky devices among multiple devices, and analyze the initial priority, resource requirements, and conflict risk coefficient of each risky task in the risky device to determine the scheduling priority of each risky task in the risky device.

[0020] The risky device is a device whose task load value exceeds the load threshold. The risky task is an unprocessed task with a base priority higher than the priority threshold. The conflict risk coefficient is used to characterize the maximum risk of resource conflict between the corresponding risky task and other tasks running simultaneously. Both the load threshold and the priority threshold are pre-stored by the scheduler; for example, the load threshold is 0.7 and the priority threshold is 0.7.

[0021] In this invention, multiple tasks are derived from the project to be tested. The base priority of each task is the base priority of the project to be tested. The scheduler can read the configuration information of the project to be tested to obtain its base priority, thereby obtaining the base priority of each task. The configuration information of the project to be tested includes at least: event type, total number of events, resource requirements, and base priority. The base priority ranges from [0,1]. A higher base priority indicates a higher urgency of the project to be tested.

[0022] For example, for each device, the scheduler can collect the device's CPU utilization and memory usage, and calculate the device's task load rate by weighting the CPU utilization and memory usage. For instance, the weight of CPU utilization can be preset to 0.6, and the weight of memory usage can be preset to 0.4.

[0023] Specifically, the step of analyzing the initial priority, resource requirements, and conflict risk coefficients of each risk task in the risk equipment to determine the scheduling priority of each risk task in the risk equipment includes: Based on preset correlation values, preset risk coefficients, and preset urgency coefficients, the basic priority of risk tasks is optimized to obtain the initial priority of risk tasks. The initial priority is proportional to the basic priority of the corresponding risk task, proportional to the preset correlation value of the corresponding risk task, proportional to the preset risk coefficient of the corresponding risk task, and proportional to the preset urgency coefficient of the corresponding risk task. Data analysis is performed based on the unit consumption baseline of the risk task, the number of events included in the risk task, and the preset risk coefficient to obtain the resource requirement of the risk task. The resource requirement of the risk task is proportional to the unit consumption baseline of the corresponding risk task, the number of events included in the corresponding risk task, and the preset risk coefficient of the corresponding risk task. The unit consumption baseline of the risk task is calculated based on the corresponding historical statistical values ​​and the basic resource quota of the project to be tested. Based on the initial priority of the risk task, the resource requirements of the risk task, the available resources of the equipment corresponding to the risk task, and the conflict risk coefficient of the risk task, data analysis is performed to obtain the scheduling priority of the risk task. The scheduling priority of the risk task is proportional to the initial priority of the risk task, and the scheduling priority of the risk task is proportional to the conflict risk coefficient of the risk task.

[0024] For example, suppose the first u The first device is a risk device. u The first of the devices j This task is a risky task, and its initial priority is... It can be represented as: In the formula, For the first u The first of the devices j The basic priority of each task YL u,j The preset correlation value is used to characterize the degree of interdependence between the corresponding task and other tasks. The preset correlation value is stored in advance by the scheduler and the value range is [0,1]. The larger the correlation value, the higher the degree of interdependence between the corresponding task and other tasks.

[0025] LF u,j This is a preset risk coefficient; the larger the preset risk coefficient, the higher the risk. For example, if the first... j If all tasks involve touch events, then the first task... j The preset risk coefficient for the first task can be 0.12; if the first task... j If all tasks involve click events, then the first task... j The preset risk coefficient for the first task can be 0.28; if the first task... j If a task includes both touch events and click events, the scheduler can weight and sum the preset risk coefficients corresponding to the touch events and the click events according to the ratio of touch events to click events to obtain the result. jThe preset risk coefficient for each task, for example, if the ratio of touch events to click events is 3:2, then the first task... j The preset risk coefficient for each task is 0.12×0.6+0.28×0.4=0.184.

[0026] GJ u,j The preset urgency coefficient is used to characterize the urgency level of the corresponding task. The larger the preset urgency coefficient, the higher the urgency level of the corresponding task. The value range of the preset urgency coefficient is [0,1]. norm ( ) represents a normalization function used to map normalized values ​​to [0,1]. For example, the normalization function can be a maximum-minimum normalization function, where the maximum and minimum values ​​are pre-stored by the scheduler. For example, the maximum value is 3 and the minimum value is 0.

[0027] For example, the first u The first of the devices j Resource requirements for any type of resource for each task D u,j,i It can be represented as: In the formula, B u,j For the first u The first of the devices j The number of events included in each task LF u,j This is a preset risk factor. This is a preset risk amplification factor, for example, the preset risk amplification factor has a value range of [0,1]. C u,j,i For the first u The first of the devices j The first task i The unit consumption benchmark for this type of resource. C u,j,i It is determined by the scheduler based on the first i The normalized historical statistics of the resource class are used to calculate the base resource quota of the tested project after normalization. For example, C u,j,i This can be the normalized historical statistical value and the normalized mean of the basic resource quota of the project under test. For example, the scheduler obtains the first... i The normalized historical statistical value of the resource class is 0.11, and the normalized basic resource quota of the project under test is 0.2. C u,j,i It can be 0.155.

[0028] The scheduler stores weights for each resource type. It can determine the resource requirement for a risky task by calculating the weighted sum of the resource requirements for each resource type. For example, resource types include CPU specifications, memory capacity, and remaining storage space. The weight for CPU specifications is 0.6, the weight for memory capacity is 0.3, and the weight for remaining storage space is 0.1.

[0029] For example, the first u The first of the devices j Scheduling priority of each task It can be represented as: In the formula, For the first u The first of the devices j The initial priority of each task. R u For the first j The first task corresponding to the u The available resources of a device, also known as the first device. j The first task corresponding to the u The remaining resources of each device, which can be collected by the scheduler. D u,j For the first u The first of the devices j Resource requirements for each task. P k For the first k The basic priority of each project. For example, a very small positive number. The value is 0.001.

[0030] CR u,j For the first u The first of the devices j The conflict risk coefficient of the first task. For example, the process of obtaining this conflict risk coefficient includes: the scheduler can obtain the conflict risk coefficient of the first task... u The first of the devices j The task and its location u The probability of resource conflicts when all risky tasks in a device run together is calculated, and the highest probability of resource conflict is denoted as . P conf_max . No. u The first of the devices j The conflict risk coefficient of a task can be expressed as: Among them, the first u The first of the devices j The task and its location u Any risk task among the devices (let's say it's the first one)q The process of obtaining the resource conflict probability for each task running together is as follows: For each type of resource, the scheduler can obtain the probability of the first task running together. j The first task and the first q The sum of resource requirements for each task is calculated, and this sum of resource requirements is compared with the sum of resource requirements for the first task. u If the sum of resource demands for a given type of resource on each device is less than or equal to the available resource quantity for that type of resource, there is no resource conflict, and the conflict probability for that type of resource is determined to be a minimum, for example, 0.001. If the sum of resource demands exceeds the available resource quantity for that type of resource, a difference is calculated between the sum of resource demands and the available resource quantity for that type of resource; the difference is the excess. Subsequently, the controller can calculate a ratio based on the excess and the sum of resource demands, and determine the ratio as the conflict probability for that type of resource. The scheduler can obtain the maximum value among the conflict probabilities of multiple resource types and determine this maximum value as the conflict probability of the [number of] devices. j The first task and the first q The probability of resource conflicts when multiple tasks run together.

[0031] In the embodiments provided by the present invention, before identifying risky devices among multiple devices, the method further includes: The throughput of the device cluster is obtained by performing feature analysis based on the processing rate of each device and the historical crash rate of each device. The device cluster is a cluster composed of all devices. The throughput of the device cluster is positively correlated with the processing rate of multiple devices and negatively correlated with the historical crash rate of multiple devices. The test item is split according to the event type to obtain multiple event streams. Each event stream corresponds to an event sequence of an event type, including at least one of touch screen events and click events. The event sequence is a sequence of multiple events of the same event type. For each event stream, feature analysis is performed based on the number of events included in each event stream, the processing rate of each device, the throughput of the device cluster, and the risk correction coefficient to obtain the number of events corresponding to each device for each event stream. Among them, the number of events corresponding to each event stream is proportional to the processing rate of the corresponding device, proportional to the number of events included in the corresponding event stream, proportional to the risk correction coefficient, and inversely proportional to the throughput of the device cluster. For each device, based on the number of events corresponding to each device in each event stream, the corresponding number of events are obtained from multiple event streams to obtain multiple events corresponding to each device; For each device, the multiple events corresponding to each device are split and processed according to the rated processing threshold of each device to obtain multiple tasks corresponding to each device. The rated processing threshold is used to characterize the maximum number of events that can be safely executed by the corresponding device in a single instance.

[0032] For example, the throughput of the device cluster F all It can be represented as: In the formula, This represents the total number of devices in the device cluster. F u For the first u Processing rate of each device For the first u The scheduler can determine the historical crash rate of each device. u The ratio of the number of crashes to the total number of executions for a given device in historical data is determined as the [number of]th [device]. u Historical crash rate of each device.

[0033] Simultaneously, the scheduler can split the test item according to event type to obtain multiple event streams, thereby calculating the number of events for each device for each event stream. For any given device, the scheduler can obtain the sum of the number of events for that device across multiple event streams, thus obtaining the total number of events for each device. For example, the first... u The total number of events corresponding to each device can be expressed as: M u : In the formula, M u0 For the first u The basic number of events for each device is the ratio of the number of events included in the test item to the number of devices. F u For the first u The processing rate of each device. F all This represents the throughput of the device cluster. This is a risk correction factor. For the first u Historical crash rate of each device. This indicates rounding down to the nearest integer.

[0034] For example, the uThe total number of events corresponding to each device is 5000. Among them, 3000 events are obtained from the event stream of touch event type, and 2000 events are obtained from the event stream of click event type. That is to say, the scheduler determines the number of events after the division. u The ratio of touch events to click events in each task on each device is 3:2. Assume the... u The rated processing threshold for each device is 2000. The scheduler can determine the number of devices. u The first task of the device includes 1200 touch events and 800 click events; the second task includes 1200 touch events and 800 click events; and the third task includes 600 touch events and 400 click events.

[0035] In one example, the steps to obtain the processing rate for each device include: Analyze the fluctuation of historical data for each type of resource to obtain the weight of each type of resource. The resource types include at least: CPU specifications, memory capacity, and remaining storage space. The target device's comprehensive performance index is obtained by weighting the attribute values ​​of multiple types of resources of the target device and the weight of each type of resource. The comprehensive performance index is used to characterize the processing capability of the corresponding device. The target device is any device in the device cluster. Based on the comprehensive performance index of the target device and the unit event consumption time of the target device in each event type, the ratio is calculated to obtain the type-specific processing rate of the target device in each event type. The target device's processing rate is obtained by weighting the processing rates of the target device under multiple event types and the proportion of each event type in the test items.

[0036] For each type of resource, the scheduler can obtain the entropy value of the historical data of that type of resource, and calculate the weight of each type of resource based on multiple entropy values. The entropy value is negatively correlated with the weight.

[0037] For example, the first i Weight of class resources It can be represented as: In the formula, n_cx The number of resource types. S i For the first i The entropy value of a resource class. For example, a very small positive number. The value is 0.001.

[0038] For theu The scheduler can collect data from the first device. u The scheduler calculates the attribute values ​​of various resources for each device at the current moment and normalizes these values ​​to [0,1]. Then, the scheduler performs a weighted summation of the normalized attribute values ​​of multiple resource types and their corresponding weights to obtain the result. u The overall performance index of each device.

[0039] For example, the first u The device is t Comprehensive performance index at any time XF t,u It can be represented as: In the formula, For the first i The weight of class resources, n_cx The number of resource types. x i For the first u The device is t The first moment i The attribute values ​​of the resource class. x i0_min The first in historical data u The first device i Minimum value of class resources, x i0_max The first in historical data u The first device i The maximum value of a resource class. Indicates the first i The normalized attribute values ​​of the resource class. x i0_min equal x i0_max In this case, the scheduler can... i The normalized attribute value of the resource class is directly set to 0.5.

[0040] In this embodiment of the invention, the scheduler can obtain the unit event time of each device under each event type based on historical data. Subsequently, the scheduler can obtain the type-specific processing rate of the target device under each event type based on the comprehensive performance index of the target device and its unit event time under each event type. For example, assume that the first... u The target device is the first device. u The first device v The categorized processing rate of each event type F u,v It can be represented as: In the formula, XFt,u For the first u The device is t The overall performance index at any given time. T u,v For the first u The first device v The unit event duration for each event type.

[0041] For example, the first u Processing rate of each device F u It can be represented as: In the formula, V This represents the total number of event types. F u,v For the first u The first device v The processing rate for each event type. Z v For the first v The percentage of the total number of events of each event type in the projects to be tested, i.e., the percentage of the total number of events of each event type. v The ratio of the total number of events of each event type to the total number of events included in the project under test.

[0042] In one example, the steps to obtain the rated processing threshold for each device include: Based on the target device’s processing rate, preset execution time and risk correction coefficient, a product operation is performed to obtain the target device’s initial rated processing threshold. The initial rated processing threshold of the target device is rounded down to obtain the rated processing threshold of the target device.

[0043] The scheduler pre-stores a preset execution duration, such as 5 minutes. Based on the preset execution duration, the target device's processing rate, and a risk correction factor, the scheduler can obtain the target device's rated processing threshold. For example, the first... u Rated processing threshold of each device B u It can be represented as: In the formula, F u For the first u The processing rate of each device. This is the risk correction factor. T 0 represents the preset execution time. This indicates rounding down to the nearest integer.

[0044] Step S2: According to the scheduling priority of each risk task in the risk device, schedule each risk task in the risk device to the task cluster of the corresponding target device in turn.

[0045] Among them, the target device is other devices whose available resources are greater than or equal to the startup resource threshold of the corresponding risk task, and the scheduling order of the risk task is positively correlated with its corresponding scheduling priority.

[0046] In other words, after determining the scheduling priority of each risk task, the scheduler determines the target device corresponding to the risk task in descending order of scheduling priority. After determining the target device, the scheduler immediately schedules the risk task to the task cluster of the corresponding target device. By prioritizing the scheduling of risk tasks with higher scheduling priority to other devices, high-priority tasks can obtain resources in a timely manner and be executed first. This can shorten the waiting time of high-schedule-priority risk tasks and improve the testing timeliness of high-schedule-priority risk tasks.

[0047] For example, the first u The first device j Resource threshold for starting a task TY u,j It can be represented as: In the formula, D u,j For the first u The first of the devices j Resource requirements for each task. XS The preset coefficient is pre-stored for the scheduler, preferably with a value of 1.2.

[0048] Step S3: Based on the utilization rate of various resources of each device's corresponding task cluster and the weight of each resource, perform weighted processing to obtain the comprehensive utilization rate of the basic resource quota of each device's corresponding task cluster.

[0049] The scheduler can collect the utilization rate of various resources for each task cluster corresponding to each device at the current moment. In other words, the scheduler can collect the utilization rate of various resources for any task cluster at the current moment. Subsequently, the scheduler performs weighted processing based on the utilization rate of various resources by the task cluster at the current moment and the weights corresponding to each resource, to obtain the comprehensive utilization rate of the basic resource quota for each task cluster corresponding to each device. The method for obtaining the weights corresponding to various resources is as described in step S1 above, and will not be repeated here.

[0050] For example, the first l A task cluster in t The overall utilization rate of basic resource quotas at all times USl ( t This can be represented as: In the formula, n_cx The number of resource types. For the first i Weights of resource classes. For the first l A task cluster in t Time for the first i The utilization rate of class resources.

[0051] In one example, the steps to obtain the basic resource quota for each task cluster include: The basic resource quota of the test project is obtained by merging the basic priority of the test project and the total resource volume of the device cluster. The basic resource quota of the test project is proportional to the basic priority of the test project and the basic resource quota of the test project is proportional to the total resource volume. Based on the proportion of the resource requirements of each task cluster to the resource requirements of the project under test, the basic resource quota of the project under test is allocated to each task cluster proportionally, thus obtaining the basic resource quota of each task cluster.

[0052] For example, suppose the first k The project is the project to be tested. k Basic resource quotas for each project R k,base It can be represented as: In the formula, R total This represents the total resources of the device cluster. P k For the first k The basic priority of each project. m This represents the total number of unprocessed items to be tested. Understandably, if the... k The basic resource quota for project number one is less than that for project number two. k The resource requirements of each project, and the first k If the base priority of a project is greater than the priority threshold (such as 0.7 mentioned above), the scheduler can preempt resources from projects whose base priority is less than or equal to the priority threshold, until the [number]th project... k The basic resource allocation for project number one is greater than that for project number two. kThe resource requirements of each project are defined. This ensures that when multiple projects compete for limited resources—that is, when the cumulative resource requirements of all unprocessed test projects exceed the total resource capacity of the device cluster—high-priority test projects can obtain resource quotas first, and if insufficient, they can be preempted to make up the shortfall. This avoids the problem of low-priority test projects occupying resources for extended periods, preventing high-priority test projects from being processed in a timely manner, thus ensuring that high-priority test projects are processed first.

[0053] Step S4: Based on the overall utilization rate of the basic resource quota for each task cluster corresponding to each device, optimize the basic resource quota for each task cluster corresponding to each device to obtain the target resource quota for each task cluster corresponding to each device.

[0054] Specifically, the optimization of the basic resource quota for each task cluster corresponding to each device based on the overall utilization rate of the basic resource quota for each task cluster, to obtain the target resource quota for each task cluster corresponding to each device, includes: By analyzing the relative magnitudes of the resource redundancy, the first redundancy threshold, and the second redundancy threshold corresponding to the target task cluster, an adjustment coefficient for the resource quota of the target task cluster is obtained. The target task cluster is any one of multiple task clusters corresponding to multiple devices. The resource redundancy is used to characterize the proportion of unused resources. The first redundancy threshold is greater than the second redundancy threshold. The adjustment coefficient of the target task cluster is calculated based on the comprehensive utilization rate of the target task cluster to the corresponding target device and a preset adjustment step size. Calculate the adjustment coefficient of the resource quota of the target task cluster, and multiply it by the basic resource quota of the target task cluster to obtain the target resource quota of the target task cluster.

[0055] The step of analyzing the relative magnitudes of the resource redundancy, the first redundancy threshold, and the second redundancy threshold corresponding to the target task cluster, and obtaining the adjustment coefficient of the resource quota for the target task cluster, includes: When the resource redundancy corresponding to the target task cluster is greater than the first redundancy threshold, the adjustment coefficient of the resource quota of the target task cluster is the difference between the first product value and the preset coefficient, wherein the first product value is the product of the idle rate and the preset adjustment step size, and the idle rate is used to characterize the proportion of idle resources. If the resource redundancy corresponding to the target task cluster is greater than the second redundancy threshold and less than or equal to the first redundancy threshold, the adjustment coefficient of the resource quota of the target task cluster is directly assigned to the preset target coefficient. When the resource redundancy corresponding to the target task cluster is less than or equal to the second redundancy threshold, the adjustment coefficient of the resource quota of the target task cluster is the sum of the second product value and the preset coefficient, wherein the second product value is the product of the overload rate and the preset adjustment step size, and the overload rate is used to characterize the proportion of overloaded resources.

[0056] For example, suppose the first l The first task cluster is the target task cluster, and the second... l A task cluster in t Target resource quota at any time R l ( t This can be represented as: In the formula, R l,base For the first l Basic resource quotas for each task cluster. For the first l A task cluster in t The adjustment coefficient at any given time. For example, the scheduler pre-stores a first redundancy threshold of 0.2, a second redundancy threshold of 0, and a preset adjustment step size of 0.1. This... l A task cluster in t The adjustment coefficient at time can be expressed as: In the formula, Y Pre-stored preset coefficients for the scheduler, preferably with a value of 1. US l ( t ) is the first l A task cluster in t The overall utilization rate of basic resource quotas at all times. 1- US l ( t () indicates the idle rate. US l ( t )-1 represents the overload rate. MY The target coefficient is pre-stored for the scheduler, preferably with a value of 1. S l ( t ) is the first l A task cluster in t The resource redundancy corresponding to a given moment, for example, S l ( t This can be represented as: In the formula, LF This is a correction factor, which is the first correction factor. l The average of the preset risk coefficients for all tasks in a task cluster. This is the preset risk amplification factor mentioned above.

[0057] In this invention, a snapshot can be taken of each completed assigned task to obtain a status record of the completed task. A snapshot refers to an instantaneous record of the current state of tasks in the device cluster. Based on this, the scheduler can determine the completed tasks according to the status record when a device crash causes task execution to be interrupted, and reassign unfinished tasks to other available devices for continued execution. This avoids a full rerun due to device crashes, achieves breakpoint resumption functionality, and improves the fault tolerance and efficiency of the test.

[0058] In summary, this invention identifies risky devices among multiple devices and analyzes the initial priority, resource requirements, and conflict risk coefficients of each risky task within the risky device to determine the scheduling priority of each risky task. Then, according to the scheduling priority of each risky task in the risky device, each risky task is sequentially scheduled to the task cluster of the corresponding target device. Subsequently, based on the utilization rate of various resources and the corresponding weights of each resource in the task cluster corresponding to each device, a weighted average is performed to obtain the comprehensive utilization rate of the basic resource quota for the task cluster corresponding to each device. Finally, based on the comprehensive utilization rate of the basic resource quota for the task cluster corresponding to each device, the basic resource quota for the task cluster corresponding to each device is adjusted. Optimization is performed to obtain the target resource quota for each task cluster corresponding to each device. On the one hand, by first identifying risky devices and then scheduling and optimizing resources for risky tasks, the problem of device overload can be alleviated, and the task interruption and re-execution caused by device downtime can be avoided, which can improve the stability of testing. On the other hand, by prioritizing the scheduling of risky tasks with higher priority to other devices, high-priority tasks can obtain resources in a timely manner and be executed first, which can improve the testing timeliness of high-priority tasks. Furthermore, by dynamically adjusting the resource quota of each task cluster through comprehensive utilization, the resource allocation is matched with the real-time load, avoiding resource idle waste or overload contention, thereby improving the resource utilization efficiency of clustered Monkey automated testing.

[0059] This invention also proposes a clustered Monkey automated testing system; please refer to [link / reference]. Figure 2 , Figure 2 This diagram illustrates the structure of a clustered Monkey automated testing system 200 according to an embodiment of the present invention. The system includes: Analysis module 201 is used to identify risky devices among multiple devices, and to analyze the initial priority, resource requirements and conflict risk coefficient of each risky task in the risky device to determine the scheduling priority of each risky task in the risky device. The risky device is a device whose task load value is greater than the load threshold, the risky task is an unprocessed task whose basic priority is higher than the priority threshold, and the conflict risk coefficient is used to characterize the maximum risk of resource conflict when the corresponding risky task runs with other tasks at the same time. The scheduling module 202 is used to schedule each risk task in the risk device to the task cluster of the corresponding target device in sequence according to the scheduling priority of each risk task in the risk device. The target device is other devices whose available resources are greater than or equal to the start resource threshold of the corresponding risk task. The scheduling order of the risk tasks is positively correlated with the scheduling priority of the risk task. The calculation module 203 is used to perform weighted processing based on the utilization rate of various resources of each device's corresponding task cluster and the weight of each resource, to obtain the comprehensive utilization rate of the basic resource quota of each device's corresponding task cluster. The optimization module 204 is used to optimize the basic resource quota of each task cluster corresponding to each device based on the comprehensive utilization rate of the basic resource quota of each task cluster corresponding to each device, so as to obtain the target resource quota of each task cluster corresponding to each device.

[0060] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the clustered Monkey automated testing system and the clustered Monkey automated testing method embodiment provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiment, which will not be repeated here.

[0061] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A cluster-based automated Monkey testing method, characterized in that, The method includes: Risk devices are identified among multiple devices, and the scheduling priority of each risk task in the risk device is determined by analyzing the initial priority, resource requirements and conflict risk coefficient of each risk task in the risk device. The risk device is a device whose task load value is greater than the load threshold, the risk task is an unprocessed task whose basic priority is higher than the priority threshold, and the conflict risk coefficient is used to characterize the maximum risk of resource conflict when the corresponding risk task runs with other tasks at the same time. According to the scheduling priority of each risk task in the risk device, each risk task in the risk device is scheduled to the task cluster of the corresponding target device in turn. The target device is other devices whose available resources are greater than or equal to the start resource threshold of the corresponding risk task. The scheduling order of risk tasks is positively correlated with the scheduling priority of the risk task. Based on the utilization rate of various resources by the task cluster corresponding to each device and the weight of each resource, the comprehensive utilization rate of the basic resource quota of the task cluster corresponding to each device is obtained. Based on the overall utilization rate of the basic resource quota for each task cluster corresponding to each device, the basic resource quota for each task cluster corresponding to each device is optimized to obtain the target resource quota for each task cluster corresponding to each device. The step of determining the scheduling priority of each risk task in the risk device by analyzing the initial priority, resource requirements, and conflict risk coefficient of each risk task in the risk device includes: Based on preset correlation values, preset risk coefficients, and preset urgency coefficients, the basic priority of risk tasks is optimized to obtain the initial priority of risk tasks. The initial priority is proportional to the basic priority of the corresponding risk task, proportional to the preset correlation value of the corresponding risk task, proportional to the preset risk coefficient of the corresponding risk task, and proportional to the preset urgency coefficient of the corresponding risk task. Data analysis is performed based on the unit consumption baseline of the risk task, the number of events included in the risk task, and the preset risk coefficient to obtain the resource requirement of the risk task. The resource requirement of the risk task is proportional to the unit consumption baseline of the corresponding risk task, the number of events included in the corresponding risk task, and the preset risk coefficient of the corresponding risk task. The unit consumption baseline of the risk task is calculated based on the corresponding historical statistical values ​​and the basic resource quota of the project to be tested. Based on the initial priority of the risk task, the resource requirement of the risk task, the available resources of the equipment corresponding to the risk task, and the conflict risk coefficient of the risk task, data analysis is performed to obtain the scheduling priority of the risk task. The scheduling priority of the risk task is proportional to the initial priority of the risk task, and the scheduling priority of the risk task is proportional to the conflict risk coefficient of the risk task. The step of optimizing the basic resource quota for each task cluster based on the overall utilization rate of the basic resource quota for each device's corresponding task cluster, to obtain the target resource quota for each device's corresponding task cluster, includes: By analyzing the relative magnitudes of the resource redundancy, the first redundancy threshold, and the second redundancy threshold corresponding to the target task cluster, an adjustment coefficient for the resource quota of the target task cluster is obtained. The target task cluster is any one of multiple task clusters corresponding to multiple devices. The resource redundancy is used to characterize the proportion of unused resources. The first redundancy threshold is greater than the second redundancy threshold. The adjustment coefficient of the target task cluster is calculated based on the comprehensive utilization rate of the target task cluster to the corresponding target device and a preset adjustment step size. Calculate the adjustment coefficient of the resource quota of the target task cluster, and multiply it by the basic resource quota of the target task cluster to obtain the target resource quota of the target task cluster.

2. The clustered Monkey automated testing method according to claim 1, characterized in that, The analysis of the relative magnitudes of the resource redundancy, the first redundancy threshold, and the second redundancy threshold corresponding to the target task cluster yields the resource quota adjustment coefficients for the target task cluster, including: When the resource redundancy corresponding to the target task cluster is greater than the first redundancy threshold, the adjustment coefficient of the resource quota of the target task cluster is the difference between the first product value and the preset coefficient, wherein the first product value is the product of the idle rate and the preset adjustment step size, and the idle rate is used to characterize the proportion of idle resources. If the resource redundancy corresponding to the target task cluster is greater than the second redundancy threshold and less than or equal to the first redundancy threshold, the adjustment coefficient of the resource quota of the target task cluster is directly assigned to the preset target coefficient. When the resource redundancy corresponding to the target task cluster is less than or equal to the second redundancy threshold, the adjustment coefficient of the resource quota of the target task cluster is the sum of the second product value and the preset coefficient, wherein the second product value is the product of the overload rate and the preset adjustment step size, and the overload rate is used to characterize the proportion of overloaded resources.

3. The clustered Monkey automated testing method according to claim 1, characterized in that, Before identifying risky devices among multiple devices, the method further includes: The throughput of the device cluster is obtained by performing feature analysis based on the processing rate of each device and the historical crash rate of each device. The device cluster is a cluster composed of all devices. The throughput of the device cluster is positively correlated with the processing rate of multiple devices and negatively correlated with the historical crash rate of multiple devices. The test item is split according to the event type to obtain multiple event streams. Each event stream corresponds to an event sequence of an event type, including at least one of touch events and click events. The event sequence is a sequence of multiple events of the same event type. For each event stream, feature analysis is performed based on the number of events included in each event stream, the processing rate of each device, the throughput of the device cluster, and the risk correction coefficient to obtain the number of events corresponding to each device for each event stream. Among them, the number of events corresponding to each event stream is proportional to the processing rate of the corresponding device, proportional to the number of events included in the corresponding event stream, proportional to the risk correction coefficient, and inversely proportional to the throughput of the device cluster. For each device, based on the number of events corresponding to each device in each event stream, the corresponding number of events are obtained from multiple event streams to obtain multiple events corresponding to each device; For each device, the multiple events corresponding to each device are split and processed according to the rated processing threshold of each device to obtain multiple tasks corresponding to each device. The rated processing threshold is used to characterize the maximum number of events that can be safely executed by the corresponding device in a single instance.

4. The clustered Monkey automated testing method according to claim 3, characterized in that, The steps to obtain the processing rate of each device include: Analyze the fluctuation of historical data for each type of resource to obtain the weight of each type of resource. The resource types include at least: CPU specifications, memory capacity, and remaining storage space. The target device's comprehensive performance index is obtained by weighting the attribute values ​​of multiple types of resources of the target device and the weight of each type of resource. The comprehensive performance index is used to characterize the processing capability of the corresponding device. The target device is any device in the device cluster. Based on the comprehensive performance index of the target device and the unit event time of the target device in each event type, the ratio is calculated to obtain the type-specific processing rate of the target device in each event type. The target device's processing rate is obtained by weighting the processing rates of the target device under multiple event types and the proportion of each event type in the test items.

5. The clustered Monkey automated testing method according to claim 4, characterized in that, The steps for obtaining the rated processing threshold for each device include: Based on the target device’s processing rate, preset execution time and risk correction coefficient, a product operation is performed to obtain the target device’s initial rated processing threshold. The initial rated processing threshold of the target device is rounded down to obtain the rated processing threshold of the target device.

6. The clustered Monkey automated testing method according to claim 1, characterized in that, The steps to obtain the basic resource quota for each task cluster include: The basic resource quota of the test project is obtained by merging the basic priority of the test project and the total resource volume of the device cluster. The basic resource quota of the test project is proportional to the basic priority of the test project and the basic resource quota of the test project is proportional to the total resource volume. Based on the proportion of the resource requirements of each task cluster to the resource requirements of the project under test, the basic resource quota of the project under test is allocated to each task cluster proportionally, thus obtaining the basic resource quota of each task cluster.

7. The clustered Monkey automated testing method according to claim 1, characterized in that, The method further includes: Take a snapshot of each assigned task after the test is completed to obtain the status record of the completed task.

8. A clustered Monkey automated testing system, characterized in that, The system includes: The analysis module is used to identify risky devices among multiple devices and analyze them based on the initial priority, resource requirements and conflict risk coefficient of each risky task in the risky device to determine the scheduling priority of each risky task in the risky device. The risky device is a device whose task load value is greater than the load threshold, the risky task is an unprocessed task with a basic priority higher than the priority threshold, and the conflict risk coefficient is used to characterize the maximum risk of resource conflict when the corresponding risky task runs with other tasks at the same time. The scheduling module is used to schedule each risk task in the risk device to the task cluster of the corresponding target device in sequence according to the scheduling priority of each risk task in the risk device. The target device is other devices whose available resources are greater than or equal to the start resource threshold of the corresponding risk task. The scheduling order of the risk tasks is positively correlated with the scheduling priority of the risk task. The calculation module is used to perform weighted processing based on the utilization rate of various resources of each device's corresponding task cluster and the weight of each resource, to obtain the comprehensive utilization rate of the basic resource quota of each device's corresponding task cluster. The optimization module is used to optimize the basic resource quota of each task cluster corresponding to each device based on the overall utilization rate of the basic resource quota of each task cluster corresponding to each device, so as to obtain the target resource quota of each task cluster corresponding to each device. The step of determining the scheduling priority of each risk task in the risk device by analyzing the initial priority, resource requirements, and conflict risk coefficient of each risk task in the risk device includes: Based on preset correlation values, preset risk coefficients, and preset urgency coefficients, the basic priority of risk tasks is optimized to obtain the initial priority of risk tasks. The initial priority is proportional to the basic priority of the corresponding risk task, proportional to the preset correlation value of the corresponding risk task, proportional to the preset risk coefficient of the corresponding risk task, and proportional to the preset urgency coefficient of the corresponding risk task. Data analysis is performed based on the unit consumption baseline of the risk task, the number of events included in the risk task, and the preset risk coefficient to obtain the resource requirement of the risk task. The resource requirement of the risk task is proportional to the unit consumption baseline of the corresponding risk task, the number of events included in the corresponding risk task, and the preset risk coefficient of the corresponding risk task. The unit consumption baseline of the risk task is calculated based on the corresponding historical statistical values ​​and the basic resource quota of the project to be tested. Based on the initial priority of the risk task, the resource requirement of the risk task, the available resources of the equipment corresponding to the risk task, and the conflict risk coefficient of the risk task, data analysis is performed to obtain the scheduling priority of the risk task. The scheduling priority of the risk task is proportional to the initial priority of the risk task, and the scheduling priority of the risk task is proportional to the conflict risk coefficient of the risk task. The step of optimizing the basic resource quota for each task cluster based on the overall utilization rate of the basic resource quota for each device's corresponding task cluster, to obtain the target resource quota for each device's corresponding task cluster, includes: By analyzing the relative magnitudes of the resource redundancy, the first redundancy threshold, and the second redundancy threshold corresponding to the target task cluster, an adjustment coefficient for the resource quota of the target task cluster is obtained. The target task cluster is any one of multiple task clusters corresponding to multiple devices. The resource redundancy is used to characterize the proportion of unused resources. The first redundancy threshold is greater than the second redundancy threshold. The adjustment coefficient of the target task cluster is calculated based on the comprehensive utilization rate of the target task cluster to the corresponding target device and a preset adjustment step size. Calculate the adjustment coefficient of the resource quota of the target task cluster, and multiply it by the basic resource quota of the target task cluster to obtain the target resource quota of the target task cluster.

Images