Test resource configuration method and device

Abstract

The invention provides a test resource configuration method and device, and the method comprises the steps: transmitting at least one test request data for a to-be-tested service in a first preset time period, determining a test response parameter corresponding to each test request data, and enabling the test request data to comprise a plurality of same or similar test request data; according to the test response parameters or statistical characteristics of the test response parameters, determining a starting state of the to-be-tested service, the starting state of the to-be-tested service being used for indicating whether cache data rapidly accessed by the to-be-tested service exists; and obtaining a preset allocation strategy corresponding to the starting state of the to-be-tested service, and configuring test resources in a second preset time period according to the preset allocation strategy.

Description

Technical Field

[0001] This application relates to the field of testing service technology, and in particular to a testing resource configuration method and apparatus. Background Technology

[0002] Caching mechanisms are widely used in cloud computing, large-scale model inference, and other fields to reduce redundant computation and shorten response latency. When a request is made to a service, the presence or absence of a response in the cache can lead to significant performance differences. Existing service testing methods typically use fixed concurrency or fixed request rates for stress testing. However, because they do not consider whether a response is in the cache, they cannot accurately simulate the gradually increasing traffic patterns and mixed scenarios of requests hitting and not hitting the cache during actual service operation, resulting in test results that deviate from the actual operating environment. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide at least one test resource configuration method and apparatus, which solves the technical problem in the prior art that the test traffic cannot be adjusted according to whether the cache is hit, resulting in a difference between the test results and the actual operating environment, thereby achieving the technical effect of improving the accuracy of testing.

[0004] This application mainly includes the following aspects: In a first aspect, embodiments of this application provide a test resource configuration method, the method comprising: sending test request data to a service to be tested within a first preset time period; determining test response parameters corresponding to each test request data, wherein the test request data includes multiple identical or similar test request data; determining the startup state of the service to be tested based on the test response parameters or statistical characteristics of the test response parameters, wherein the startup state of the service to be tested is used to indicate whether there is cached data for fast access of the service to be tested; obtaining a preset allocation strategy corresponding to the startup state of the service to be tested; and configuring test resources according to the preset allocation strategy within a second preset time period.

[0005] Optionally, the startup state of the service under test includes a cold start state and a warm start state. The cold start state indicates that there is no cached data that the service under test can quickly access, and the warm start state indicates that there is cached data that the service under test can quickly access. The startup state of the service under test is determined by: determining a target response time interval into which at least one target test response parameter and / or the statistical characteristics of the target test response parameter fall; the target response time interval includes a first response time interval, a second response time interval, and a third response time interval; the lower limit of the third response time interval is not less than the upper limit of the first response time interval, and the upper limit of the third response time interval is not greater than the lower limit of the second response time interval; when the target response time interval is the first response time interval, the startup state of the service under test is the warm start state; when the target response time interval is the second response time interval, the startup state of the service under test is the cold start state; when the target response time interval is the third response time interval, the startup state of the service under test is determined based on the historical cache parameters and / or service operation parameters of the test request data.

[0006] Optionally, the first response time interval, the second response time interval, and the third response time interval corresponding to the service under test are constructed in the following manner: real response times or statistical characteristics of each real response time corresponding to multiple real request data, and the startup status of the service under test corresponding to each real response time, wherein each real response time includes at least one first real response time in a hot start state and at least one second real response time in a cold start state; sorting the first real response time and the first statistical characteristic, and the second real response time and the second statistical characteristic, respectively; interpolating the upper limit of the third response time interval based on the first quantile of the sorted first real response time sequence or the first statistical characteristic; and interpolating the lower limit of the third response time interval based on the second quantile of the sorted second real response time sequence or the second statistical characteristic.

[0007] Optionally, the historical cache parameters include the hash value and cache lifecycle of cached data in the cache database, and the service operation parameters include at least the server queue length or resource utilization rate. Based on the historical cache parameters and / or service operation parameters of the test request data, the startup state of the service to be tested is determined, including: matching the hash value of the test request data with the hash values ​​of each cached data in the cache database; when a hash value matching the hash value of the test request data exists in the cache database, the startup state of the service to be tested is the hot start state; when no matching hash value exists in the cache database, if the sending time parameter of the previous identical test request data and the currently sent test request data does not meet the auxiliary judgment condition, the startup state of the service to be tested is the cold start state; if the sending time parameter of the previous identical test request data and the currently sent test request data meets the auxiliary judgment condition, the startup state of the service to be tested is determined by auxiliary judgment using the server queue length and / or the resource utilization rate.

[0008] Optionally, the startup state of the service under test can be determined by auxiliary judgment based on the server queue length and / or the resource utilization rate, including: when the server queue length is greater than a preset queue length threshold, and / or the resource utilization rate is greater than a preset resource utilization rate threshold, the startup state of the service under test is the hot start state; when the server queue length is less than or equal to the preset queue length threshold, and the resource utilization rate is less than or equal to the preset resource utilization rate threshold, the startup state of the service under test is the cold start state.

[0009] Optionally, the following methods can be used to determine whether the auxiliary judgment conditions are met:

[0010] in, The probability density function represents the first true response time series or the first statistical characteristic. The probability density function represents the second true response time series or the second statistical characteristic, where F is the control factor and t refers to the response time or the statistical characteristic of the response time.

[0011]

[0012] TH is the judgment parameter corresponding to the auxiliary judgment condition. It is the difference in transmission time. The standard deviation represents the first true response time series or the first statistical characteristic. The standard deviation of the second true response time series or the second statistical characteristic is represented by C. and The maximum value; if TTL≤TH, the auxiliary judgment condition is not met; if TTL>TH, the auxiliary judgment condition is met. TTL represents the cache lifecycle corresponding to the previous request data that meets the similarity condition with the service under test.

[0013] Optionally, within the second preset time period, test resources are configured according to the preset allocation strategy, including: dividing the second preset time period into an initial stage, an intermediate stage, and an end stage based on the preset allocation strategy; and configuring test traffic at each moment of the initial stage, the intermediate stage, and the end stage through the stage allocation sub-strategies corresponding to the initial stage, the intermediate stage, and the end stage in the preset allocation strategy.

[0014] Optionally, the test traffic at each moment in the second preset time period can be calculated using the following formula:

[0015] in, This refers to the number of test request data at time t in the second preset time period when the startup state of the service to be tested is a hot start state; This refers to the flow control coefficient corresponding to the hot start state. , This refers to the maximum amount of test request data that the service can stably process during the hot start state. and All are positive numbers not greater than 1; This refers to the initial stage corresponding to the hot start state; This refers to the intermediate stage corresponding to the hot start state; This refers to the ending stage corresponding to the hot start state; This refers to the preset exponent. ; , For preset peak value, The minimum value is set to .

[0016] Optionally, the test traffic at each moment in the second preset time period can be calculated using the following formula:

[0017] in, This refers to the number of test request data at time t in the second preset time period when the startup state of the service to be tested is cold start state; This refers to the flow control coefficient corresponding to the cold start state. , This refers to the maximum amount of test request data that the service can stably process during the cold start state. and All are positive numbers not greater than 1; This refers to the initial stage corresponding to the cold start state; This refers to the intermediate stage corresponding to the cold start state; This refers to the ending stage corresponding to the cold start state.

[0018] Secondly, embodiments of this application also provide a test resource configuration device, the device comprising: a response parameter determination module, which sends test request data to a service under test within a first preset time period and determines test response parameters corresponding to each test request data, wherein the test request data includes multiple identical or similar test request data; a startup status determination module, which determines the startup status of the service under test based on the test response parameters or statistical characteristics of the test response parameters, wherein the startup status of the service under test is used to indicate whether there is cached data for quick access by the service under test; and a resource configuration module, which obtains a preset allocation strategy corresponding to the startup status of the service under test and configures test resources according to the preset allocation strategy within a second preset time period.

[0019] This application provides a test resource configuration method and apparatus. The method includes: sending test request data to a service under test within a first preset time period; determining test response parameters corresponding to each test request data, wherein the test request data includes multiple identical or similar test request data; determining the startup state of the service under test based on the test response parameters or their statistical characteristics, wherein the startup state of the service under test indicates whether there is cached data that the service under test can quickly access; obtaining a preset allocation strategy corresponding to the startup state of the service under test; and configuring test resources according to the preset allocation strategy within a second preset time period. By sending test request data to the service under test and receiving test response parameters, and determining the startup state of the service under test based on the test response parameters or their statistical characteristics, and configuring test resources according to the preset allocation strategy corresponding to the startup state, this method solves the technical problem in the prior art where test traffic cannot be adjusted based on whether the cache is hit, leading to differences between test results and the actual operating environment, thereby improving test accuracy.

[0020] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart of a test resource configuration method provided in an embodiment of this application is shown.

[0023] Figure 2 This paper illustrates a functional block diagram of a test resource configuration device provided in an embodiment of this application.

[0024] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0026] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0027] In existing technologies, service performance is measured by sending requests to the service and receiving corresponding responses. However, the presence of cached responses is not considered during the testing process. Consequently, test results may differ from actual service performance, thus affecting the accuracy of the tests.

[0028] Based on this, this application provides a test resource configuration method and apparatus. By sending test request data to the service under test and determining the test response parameters, the startup state of the service under test is determined through the test response parameters or their statistical characteristics. Test resources are then configured according to a preset allocation strategy corresponding to the startup state. This solves the technical problem in the prior art where test traffic cannot be adjusted based on whether the cache is hit, leading to discrepancies between test results and the actual operating environment. This achieves the technical effect of improving test accuracy, as detailed below: Please see Figure 1 , Figure 1 This is a flowchart illustrating a test resource configuration method provided in an embodiment of this application. Figure 1 As shown, the test resource configuration method provided in this application embodiment includes the following steps: S101: Send test request data for the service to be tested within the first preset time period, and determine the test response parameters corresponding to each test request data.

[0029] In other words, within a first preset time period, multiple identical test request data or multiple similar test request data are sent sequentially to the service under test, so as to receive the test response data corresponding to each test request data and determine the receiving time corresponding to each test response data.

[0030] Furthermore, test response parameters are used to reflect the service's operational capability in response to test requests, and these parameters must include at least the response time. Response time refers to the time difference between the time it takes to send the test request data and the time it takes to receive the corresponding test response data.

[0031] The test request data includes multiple identical or similar test request data. Specifically, similar test request data includes different test request data of the same type, or test request data of similar types. Furthermore, the multiple test request data should be request data that the service will actually receive during actual operation.

[0032] For example, multiple test request data of the same type can all be query requests but query different content. This application does not limit the specific content of the test request data.

[0033] S102: Determine the startup status of the service to be tested based on the test response parameters or the statistical characteristics of the test response parameters.

[0034] The startup status of the service under test indicates whether there is cached data that can be quickly accessed by the service under test. The startup status includes a cold start status and a warm start status. A cold start status indicates that there is no cached data that can be quickly accessed by the service under test, while a warm start status indicates that there is cached data that can be quickly accessed by the service under test.

[0035] In other words, the startup status of the service under test reflects whether the physical storage area of ​​the cached data contains test response data for the test request. Furthermore, the response time or statistical characteristics of the response time are used to measure whether the physical storage area of ​​the cached data contains test response data.

[0036] For example, the number of test request data should be controlled between 3 and 10. By sending test request data multiple times and determining the statistical characteristics of the test request data, the problem of inaccurate test results caused by occasional situations can be effectively avoided. However, if too many times the data is sent repeatedly, it may also interfere with the traffic and results of subsequent tests.

[0037] Specifically, if the physical storage area of ​​the cached data contains test response data for the test request data, the service under test can directly retrieve the test response data from the physical storage area, and the response time should be short, meaning the service under test is in a warm start state. If the physical storage area of ​​the cached data does not contain test response data for the test request data, the service under test cannot directly retrieve the test response data from the physical storage area, and the response time should be long, meaning the service under test is in a cold start state.

[0038] For example, the physical storage area of ​​cached data includes: cached data residing in the memory of the service under test, or cached data residing in the middleware server (such as a high-performance key-value storage database like Redis) of the service under test, or cached data residing in the cache mechanism of the server used by the service under test.

[0039] The startup state of the service under test is determined as follows: A target response time interval is identified into which at least one target test response parameter and / or the statistical characteristics of the target test response parameter fall. This target response time interval includes a first response time interval, a second response time interval, and a third response time interval. The lower limit of the third response time interval is not less than the upper limit of the first response time interval, and the upper limit of the third response time interval is not greater than the lower limit of the second response time interval. When the target response time interval is the first response time interval, the startup state of the service under test is the warm start state. When the target response time interval is the second response time interval, the startup state of the service under test is the cold start state. When the target response time interval is the third response time interval, the startup state of the service under test is determined based on the historical cache parameters and / or service operation parameters of the test request data. Preferably, the first response time interval, the third response time interval, and the second response time interval are continuous, i.e., the lower limit of the third response time interval is equal to the upper limit of the first response time interval, and the upper limit of the third response time interval is equal to the lower limit of the second response time interval.

[0040] Specifically, the response times are arranged according to the order in which the test response data corresponding to each test request is received. For example, the target test response parameter refers to either the first or last response time obtained by sorting the response times according to their receiving times. That is, if the first response time after sorting is used as the target test response parameter, then the target test response parameter is the response time corresponding to the first received test response data; if the last response time after sorting is used as the target test response parameter, then the target test response parameter is the response time corresponding to the last received test response data.

[0041] The target test response parameter statistical characteristics refer to the statistical quantities obtained by processing the response times corresponding to each test request data. These characteristics can be the average or median of each response time. Furthermore, the startup status of the service under test should be determined based on the target response time interval falling into one of the following: the first response time, the last response time, the average of all response times, or the median.

[0042] For example, the first response time interval refers to a time less than or equal to a first time threshold, the second response time interval refers to a time greater than or equal to a second time threshold, and the third response time interval refers to a time greater than the first time threshold and less than the second time threshold. The first time threshold is used to characterize a warm start state, and the second time threshold is used to characterize a cold start state.

[0043] Therefore, the response time (RT) for a cold start is greater than that for a warm start. If the target response parameters or their statistical characteristics fall within the first response time interval, the response time is shorter, indicating that the cache contains the test response data corresponding to the sent test request data; thus, the service under test is considered to be in a warm start state. Conversely, if the target response parameters or their statistical characteristics fall within the second response time interval, the response time is longer, indicating that the cache does not contain the test response data corresponding to the sent test request data; thus, the service under test is considered to be in a cold start state.

[0044] Specifically, when the target response parameters or statistical characteristics of the target test response parameters fall within the third response time interval, the response time is at the median value. The cache may contain test response data corresponding to the sent test request data, but factors such as network fluctuations, system fluctuations, and long storage queues may cause the response time for a warm start to be longer, exceeding the upper limit of the first response time interval. Conversely, the cache may not contain the test response data corresponding to the sent test request data, but due to faster storage resource response, the response time is shortened, falling below the lower limit of the second response time interval. Therefore, the startup state of the service under test corresponding to the third response time interval may be either a cold start or a warm start, requiring the determination of the startup state of the service under test through historical cache parameters and / or service operation parameters corresponding to the test request data.

[0045] Specifically, the first response time interval, the second response time interval, and the third response time interval corresponding to the service under test are constructed in the following manner: The actual response times or statistical characteristics of each actual response time corresponding to multiple real request data are obtained, as well as the startup status of the service under test corresponding to each actual response time. Each actual response time includes at least one first actual response time in a hot start state and at least one second actual response time in a cold start state. The first actual response time and the first statistical characteristic, and the second actual response time and the second statistical characteristic are sorted respectively. The upper limit of the third response time interval is calculated by interpolation based on the first quantile of the sorted first actual response time sequence or the first statistical characteristic. The lower limit of the third response time interval is calculated by interpolation based on the second quantile of the sorted second actual response time sequence or the second statistical characteristic.

[0046] Among them, real request data refers to the request data received during the actual operation of the same or similar service to the service under test. This is used to obtain the real response time corresponding to multiple real request data, and to determine whether the cache contains the corresponding real response data when each real request data is issued, thereby determining the startup status of the service under test corresponding to each real response time.

[0047] For example, the first true response time series refers to the sorting of each true response time in the warm start state according to the data size, which can be sorted in ascending or descending order; the second true response time series refers to the sorting of each true response time in the cold start state according to the data size, which can be sorted in ascending or descending order.

[0048] For example, multiple real request data are distinguished according to their startup state as either warm start or cold start. Real request data in the warm start state is designated as first real request data, and real request data in the cold start state is designated as second real request data. The multiple first real request data in the warm start state are randomly divided into multiple first real request data groups, each group including at least two first real request data in the warm start state. Then, a first statistical feature is calculated for the multiple first real request data in each group, and the groups are sorted according to the size of the first statistical feature. Similarly, the multiple second real request data in the cold start state are randomly divided into multiple second real request data groups, each group including at least two second real request data in the cold start state. Then, a second statistical feature is calculated for the multiple second real request data in each group, and the groups are sorted according to the size of the second statistical feature. The statistical feature can be the median, the mean, or a preset order after sorting the statistical features.

[0049] For example, the first true response time is sorted in ascending order to obtain the first true response time series, and the second true response time is sorted in ascending order to obtain the second true response time series. The first quantile can be set as the 10th quantile, and the second quantile can be set as the 90th quantile. Then, the 10th quantile of the sorted first true response time series is used as the first time threshold to distinguish the hot start state, that is, the 10th quantile of the first true response time series is used as the lower limit of the third response time interval, and the 90th quantile of the sorted second true response time series is used as the second time threshold to distinguish the cold start state, that is, the 90th quantile of the second true response time series is used as the upper limit of the third response time interval. If we sort the first true response time in descending order to obtain the first true response time series, and sort the second true response time in descending order to obtain the second true response time series, the first quantile can be set as the 90th quantile, and the second quantile can be set as the 10th quantile. Then, the 90th quantile of the sorted first true response time series is used as the first time threshold to distinguish the hot start state, that is, the 90th quantile of the first true response time series is used as the lower limit of the third response time interval, and the 10th quantile of the sorted second true response time series is used as the second time threshold to distinguish the cold start state, that is, the 10th quantile of the second true response time series is used as the upper limit of the third response time interval.

[0050] For example, the sum of the first quantile and the second quantile is 1, such as the first quantile being 10% and the second quantile being 90%; or the first quantile being 5% and the second quantile being 95%.

[0051] Specifically, if the target response interval into which the target test response parameters fall is used to confirm whether the service under test is in a hot start or cold start state, then the first time threshold and the second time threshold are determined according to the first real response time sequence and the second real response time sequence obtained by sorting the real response times; if the target response interval into which the target test response parameters fall is used to confirm whether the service under test is in a hot start or cold start state, then the statistical characteristics of the real response time need to be calculated and sorted to obtain the first time threshold and the second time threshold.

[0052] If the order determined by the first quantile and the second quantile is an integer, the response time or statistical characteristic corresponding to the integer order of the quantiles is directly used as the determined time threshold; if the order determined by the first quantile and the second quantile is not an integer, the time threshold is determined by interpolation.

[0053] Specifically, if the sorted true response time or the statistical characteristics of the true response time are N, the quantile is n%, and the product of N and n% is not an integer, the time threshold is calculated using the following formula: (1) In formula (1), This refers to the product of N and n%, where y is the calculated time threshold. This refers to the order of the first integer x in the ranking of the actual response time or the statistical characteristics of the actual response time. This refers to the order of arrangement. The actual response time or the statistical characteristics of the actual response time. This refers to the order of the integer following x in the ranking of the true response time or the statistical characteristics of the true response time. This refers to the order of arrangement. The actual response time or the statistical characteristics of the actual response time.

[0054] For example, if the second true response times are sorted in ascending order to obtain the second true response time series, and the number of second true response times is 100 and the second quantile is 90%, then the 90th second true response time in the second true response time series is taken as the second time threshold; when the number of second true response times is 11 and the second quantile is 90%, since 90% × 11 = 9.9 and 9.9 is between the 9th and 10th, if the true response time of the 9th sample in the second true response time series is 8 and the true response time of the 10th sample in the second true response time series is 9.5, then the difference calculation interpolation is performed using formula (1), that is... .

[0055] Specifically, the startup state of the service to be tested is determined based on the historical cache parameters and / or service operation parameters of the test request data. This includes: matching the hash value of the test request data with the hash values ​​of each cached data in the cache database; when a hash value matching the hash value of the test request data exists in the cache database, the startup state of the service to be tested is the hot start state; when no matching hash value exists in the cache database, if the sending time parameter of the previous identical test request data and the currently sent test request data does not meet the auxiliary judgment condition, the startup state of the service to be tested is the cold start state; if the sending time parameter of the previous identical test request data and the currently sent test request data meets the auxiliary judgment condition, the startup state of the service to be tested is determined by auxiliary judgment using the server queue length and / or the resource utilization rate.

[0056] The historical cache parameters include the hash value of the cached data in the cache database and the cache lifetime. The cache lifetime (TTL, Time-to-Live) refers to the expiration time of cached data. In other words, each piece of cached data has an expiration time recorded in the cache database. When the expiration time is 0, it means that the cached data needs to be cleared from the cache database. Therefore, the shorter the cache lifetime, the shorter the effective retention time of the cached data corresponding to the requested data in the database. The service operation parameters include at least the server queue length or resource utilization rate.

[0057] Specifically, when the target test response parameters fall within the third response time interval, the hash value of the test request data corresponding to the target test response parameters is matched with the hash values ​​of each cached data in the cache database. When the statistical characteristics of the target test response parameters fall within the third response time interval, the test request data is arranged chronologically according to the sending time, and the hash value of the test request data arranged in a preset position (e.g., the first test request data) is matched with the hash values ​​of each cached data in the cache database. If the hash values ​​match, the service under test is considered to be in a warm start state. If the hash values ​​do not match, the cache database may or may not store the test request data, requiring further confirmation through auxiliary judgment conditions.

[0058] Specifically, when no matching hash value exists in the cache database, the preceding test request data that is identical to the test request data corresponding to the target test response parameters is identified, or the preceding test request data that is identical to the test request data arranged in a preset position is identified, and the corresponding cache lifecycle acquired when the preceding test request data was sent is determined. The sending time difference between the preceding test request data and the currently sent test request data is calculated, and the sending time difference and cache lifecycle are used to determine whether the auxiliary judgment condition is met. Furthermore, if the auxiliary judgment condition is not met, the startup state of the service under test is determined to be a cold start state; if the auxiliary judgment condition is met, the startup state of the service under test is determined by auxiliary judgment based on the server queue length and / or resource utilization.

[0059] The following methods are used to determine whether the auxiliary judgment conditions are met: (2) In formula (2), The probability density function represents the first true response time series or the first statistical characteristic. The probability density function represents the second true response time series or the second statistical characteristic, where F is the control factor and t represents the response time or the statistical characteristic of the response time. (3) (4) In formulas (3) and (4), TH is the judgment parameter corresponding to the auxiliary judgment condition. It is the difference in transmission time. The standard deviation represents the first true response time series or the first statistical characteristic. The standard deviation of the second true response time series or the second statistical characteristic is represented by C. and The maximum value of TTL; if TTL≤TH, the auxiliary judgment condition is not met; if TTL>TH, the auxiliary judgment condition is met. TTL represents the cache lifecycle corresponding to the previous request data that meets the similarity condition with the service under test. Here, meeting the similarity condition can refer to historical request data whose similarity to the request data of the service under test meets the preset similarity condition. The similarity is obtained by calculating the semantic similarity between the request data of the service under test and the historical request data. For example, by transforming each request data into the latent space, we obtain several historical embedding vectors corresponding to historical request data and the test embedding vector corresponding to the service under test. The historical embedding vector whose cosine similarity with the test embedding vector meets the minimum similarity threshold is taken as the target vector. The latest request data in the historical request data corresponding to the target vector is taken as the previous request data that meets the similarity condition.

[0060] In other words, if the control factor F approaches 1, it means that the startup state of the service under test is close to the warm start state, and the judgment of the cache lifecycle will be tightened; if the control factor F approaches 0, it means that the startup state of the service under test is close to the cold start state, and the judgment of the cache lifecycle will be looser.

[0061] Furthermore, a probability density function is calculated based on the first true response time series or first statistical characteristic corresponding to the warm start state, and a probability density function is calculated based on the second true response time series or second statistical characteristic corresponding to the cold start state. This allows for the calculation of a control factor using the response time or statistical characteristics of the response time in real-world warm and cold start states. The difference in transmission time between the previous test request data and the currently sent test request data is calculated, along with the first standard deviation of the first true response time series or the sorted first statistical characteristic, and the second standard deviation of the second true response time series or the sorted second statistical characteristic. The maximum value between the first and second standard deviations is taken. A judgment parameter is calculated using the maximum value, the transmission time difference, and the control factor. This judgment parameter is then compared with the cache lifetime corresponding to the service under test. If the judgment parameter is greater than or equal to the cache lifetime corresponding to the service under test, the auxiliary judgment condition is met. If the judgment parameter is less than the cache lifetime corresponding to the service under test, the auxiliary judgment condition is not met, and the service under test is considered to be in a cold start state.

[0062] Specifically, the startup state of the service under test is determined by auxiliary judgment based on the server queue length and / or the resource utilization rate, including: when the server queue length is greater than a preset queue length threshold, and / or the resource utilization rate is greater than a preset resource utilization rate threshold, the startup state of the service under test is the warm start state; when the server queue length is less than or equal to the preset queue length threshold, and the resource utilization rate is less than or equal to the preset resource utilization rate threshold, the startup state of the service under test is the cold start state.

[0063] In other words, when determining whether the parameter is greater than or equal to the cache lifecycle of the service under test, it is also necessary to compare the server queue length with the preset queue length threshold and the resource utilization rate with the preset resource utilization rate threshold to determine whether the startup state of the service under test is a hot start or a cold start.

[0064] Specifically, a longer server-side queue length will result in a longer response time even if the cache is hit; similarly, higher resource utilization will also lead to a longer response time even if the cache is hit. Therefore, when the target test response parameters or their statistical characteristics are within the third response time interval but only slightly higher than the first time threshold, a longer server-side queue length and / or higher resource utilization may indicate a cache hit. Conversely, when the target test response parameters or their statistical characteristics are within the third response time interval but only slightly lower than the second time threshold, and the queue length is short with low resource utilization, it may indicate a cache miss.

[0065] In other words, when the auxiliary judgment conditions are met, if either the server queue length is greater than the preset queue length threshold or the resource utilization rate is greater than the preset resource utilization rate threshold, the startup state of the service to be tested is determined to be a warm start state; if both the server queue length is less than or equal to the preset queue length threshold and the resource utilization rate is less than or equal to the preset resource utilization rate threshold, the startup state of the service to be tested is determined to be a cold start state.

[0066] S103: Obtain the preset allocation strategy corresponding to the startup status of the service to be tested, and configure test resources according to the preset allocation strategy within a second preset time period.

[0067] The preset allocation strategies include a first allocation strategy when the service under test is in a warm start state, and a second allocation strategy when the service under test is in a cold start state. The traffic allocation trends of the first and second allocation strategies are opposite. Therefore, when the service under test is determined to be in a warm start state, the preset allocation strategy is the first allocation strategy; when the service under test is determined to be in a cold start state, the preset allocation strategy is the second allocation strategy. Regardless of whether the service under test is in a cold start or warm start state, the overall test traffic should show an upward trend. For a cold start state, the increase in test traffic is slower; for a warm start state, the increase in test traffic is faster.

[0068] Specifically, the allocation strategy is used to adjust the test resources for testing the service under test. The test resources include at least one of the following: total number of requests, request rate, request distribution, and request traffic. The request rate includes: query per second (QPS) and / or concurrency. The request distribution includes: different request types, different users, and different data proportions. The request traffic represents the number of requests sent per unit of time.

[0069] For services that are highly sensitive to caching, such as search, recommendation, and model inference, their performance is heavily dependent on cache hit rate. Stress testing based solely on a fixed volume of traffic can yield incomplete results. For instance, when traffic is low, the cache is empty, resulting in a low hit rate. As requests increase, the cache gradually fills up, and the hit rate rises. This rising cache volume simulates the entire process of a cache going from cold start to stable hits. Once the cache is full, the cache replacement strategy can be observed, allowing us to monitor whether the hit rate fluctuates and whether cache breakdown or cache avalanche occurs. In real-world service environments, traffic typically increases continuously in an upward trend, rather than instantaneously filling up. This approach better simulates actual business operations.

[0070] Specifically, the cache replacement strategies include LRU (Least Recently Used) and LFU (Least Frequently Used). LRU focuses on the time dimension, assuming that recently used data is likely to be used in the future, while data that has not been used for a long time may no longer be needed. LFU, on the other hand, focuses on the usage frequency dimension, assuming that frequently accessed data is more valuable, while infrequently accessed data can be discarded. Cache penetration refers to a highly popular cache key suddenly expiring at a certain moment, causing a large number of requests to directly hit the database or backend, leading to the database or backend crashing due to the sudden surge in data volume. Cache avalanche refers to a large number of cache keys expiring simultaneously, resulting in a large-scale origin server request surge, which can also cause the database or backend to crash instantly due to the surge in data volume.

[0071] In this scenario, regardless of whether the service under test is in a hot start or cold start state, treating it as a hot start with rapidly increasing request traffic can easily lead to unresponsiveness when the service is in a cold start state. Conversely, treating it as a cold start with slowly increasing request traffic can result in underutilization of the service during a hot start, failing to test its capacity limits. Setting the test traffic to an intermediate trend underestimates both hot and cold start scenarios, leading to incomplete testing. Therefore, for hot start scenarios, a steeper growth curve is needed to test the cache's capacity limits. For cold start scenarios, because the resource consumption is significantly higher than hot start, a flatter growth curve is required to prevent server or backend crashes and to observe stability performance and cache degradation. Furthermore, the same abnormal behavior may be caused by different reasons in cold and hot start scenarios.

[0072] Specifically, within the second preset time period, test resources are configured according to the preset allocation strategy, including: dividing the second preset time period into an initial stage, an intermediate stage, and an end stage based on the preset allocation strategy; and configuring test traffic at each moment of the initial stage, intermediate stage, and end stage through the stage allocation sub-strategies corresponding to the initial stage, the intermediate stage, and the end stage in the preset allocation strategy.

[0073] In other words, whether it is the first allocation strategy or the second allocation strategy, it is necessary to divide the second preset time period into an initial stage, an intermediate stage and an end stage. The duration of the stages may be different for different allocation measurements, and different allocation sub-strategies are configured for each stage, so as to configure the test traffic according to the allocation sub-strategies corresponding to each stage.

[0074] For example, the first partition time used to distinguish between the initial and intermediate phases, and the second partition time used to distinguish between the intermediate and final phases, can be initially set manually and then adjusted based on service performance. Cold start and warm start states can correspond to different first and second partition times. For instance, the initial phase duration can be set to 15% of the second preset time period, the intermediate phase duration to 60% of the second preset time period, and the final phase duration to 25% of the second preset time period. That is, the first partition time is 15% of the second preset time period, and the second partition time is 75% of the second preset time period. After the initial experiment, observe the performance trend. If key indicators (such as resource utilization and response time) have stabilized well before the first partition time arrives, the first partition time can be appropriately reduced. Similarly, if key indicators are still fluctuating significantly at or after the first partition time, extend T1, and the moment when the indicators stabilize can be used as the first partition time. If key indicators deteriorate well before the second partition time arrives, such as increased response time and error rate, or consistently high resource utilization, the second partition time should be reduced; that is, the time when performance begins to decline should be used as the second partition time.

[0075] Specifically, regarding the phase division of the distribution measurement corresponding to the hot start state, if the performance begins to degrade within a very short time, the flow control coefficient corresponding to the hot start state should be reduced. and preset power If in If it remains stable for a period of time afterward, then increase And consider whether it is necessary to increase the flow control factor. and preset power The same principle applies to the phase division of the allocation measurement corresponding to the cold start state.

[0076] Specifically, the test traffic at each moment in the second preset time period is calculated using the following formula: (5) In formula (5), This refers to the number of test request data at time t in the second preset time period when the startup state of the service to be tested is a hot start state; This refers to the flow control coefficient corresponding to the hot start state. , This refers to the maximum amount of test request data that the service can stably process during the hot start state. and All are positive numbers not greater than 1; This refers to the initial stage corresponding to the hot start state; This refers to the intermediate stage corresponding to the hot start state; This refers to the ending stage corresponding to the hot start state; This refers to the preset exponent. ; , For preset peak value, The minimum value is set to .

[0077] In other words, The first distinguishing time is used to differentiate between the initial and intermediate stages corresponding to the hot start state. The second time partition is used to distinguish between the intermediate and final stages corresponding to the hot start state. Furthermore, in... This is the preheating phase; a small number of duplicate requests should be sent to ensure the cache is loaded completely. During the acceleration phase, the request sending rate should be gradually increased, and the rate at which hotspot rates are hit should be simulated to increase, with an ever-increasing number of hits. This is a rapid growth phase, characterized by exponential growth.

[0078] Among them, the preset peak value It can be manually set, that is, the highest query per second (QPS) that the developer wants the service to achieve in a hot start state; or, a preset peak value. It can be dynamically modified, for example, by first setting a preset peak value. for This is equal to the maximum query rate per second under hot start conditions. To conduct further testing, continuously increase the preset peak value. .

[0079] Specifically, the test traffic at each moment in the second preset time period is calculated using the following formula: (6) In formula (6), This refers to the number of test request data at time t in the second preset time period when the startup state of the service to be tested is cold start state; This refers to the flow control coefficient corresponding to the cold start state. , This refers to the maximum amount of test request data that the service can stably process during the cold start state. and All are positive numbers not greater than 1; This refers to the initial stage corresponding to the cold start state; This refers to the intermediate stage corresponding to the cold start state; This refers to the ending stage corresponding to the cold start state.

[0080] In other words, The first distinguishing time is used to differentiate between the initial and intermediate stages corresponding to the cold start state. The second time partition is used to distinguish between the intermediate and final stages of the cold start state. Furthermore, This is the warm-up phase, starting with a small number of requests and keeping traffic at a low, linear rate. This is a phase of slower growth. The simulation should depict scenarios where increased request volume is coupled with limited throughput due to constraints on the Central Processing Unit (CPU) or input / output capabilities. This is a phase where traffic is approaching saturation. It maintains a slow increase to avoid cache breakdown. It simulates a state where a small amount of new traffic still enters after the system has reached its processing saturation state. It is suitable for long-term observation of the system's stability performance.

[0081] For example, m takes a value of 0.1, n takes a value of 1, i takes a value of 0.05, and j takes a value of 0.15. Here, the flow control coefficient corresponds to the hot start state. Flow control coefficient corresponding to cold start state The value of can directly determine the baseline and growth level of the query rate per second (QPS). This can be determined by measuring the maximum query rate per second that the service can stably handle under the conditions of a warm start and a cold start by performing baseline performance tests.

[0082] Based on the same application concept, this application also provides a test resource configuration device corresponding to the test resource configuration method provided in the above embodiments. Since the principle of the device in this application is similar to the test resource configuration method in the above embodiments of this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0083] For example, Figure 2 This is a functional block diagram of a test resource configuration device provided in an embodiment of this application. Figure 2As shown, the test resource configuration device 10 includes: a response parameter determination module 101, which sends test request data to the service under test within a first preset time period and determines the test response parameters corresponding to each test request data, wherein the test request data includes multiple identical or similar test request data; a startup status determination module 102, which determines the startup status of the service under test based on the test response parameters or the statistical characteristics of the test response parameters, wherein the startup status of the service under test is used to indicate whether there is cached data for fast access of the service under test; and a resource configuration module 103, which obtains a preset allocation strategy corresponding to the startup status of the service under test and configures test resources according to the preset allocation strategy within a second preset time period.

[0084] Based on the same application concept, see [link / reference] Figure 3 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. The electronic device 20 includes a processor 201, a memory 202, and a bus 203. The memory 202 stores machine-readable instructions executable by the processor 201. When the electronic device 20 is running, the processor 201 and the memory 202 communicate through the bus 203. The machine-readable instructions are executed by the processor 201 to perform the steps of the test resource configuration method described in any of the above embodiments.

[0085] Specifically, when the machine-readable instructions are executed by the processor 201, they can perform the following processing: sending at least one test request data to the service under test within a first preset time period; determining the test response parameters corresponding to each test request data, wherein the test request data includes multiple identical or similar test request data; determining the startup state of the service under test based on the test response parameters or the statistical characteristics of the test response parameters, wherein the startup state of the service under test is used to indicate whether there is cached data for fast access of the service under test; obtaining a preset allocation strategy corresponding to the startup state of the service under test; and configuring test resources according to the preset allocation strategy within a second preset time period.

[0086] Based on the same concept, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the test resource configuration method provided in the above embodiments.

[0087] Specifically, the storage medium can be a general-purpose storage medium, such as a portable disk or hard disk. When the computer program on the storage medium is run, it can execute the above-mentioned test resource configuration method. By sending test request data to the service under test and receiving test response parameters, the startup status of the service under test can be determined by the test response parameters or their statistical characteristics. Test resources can then be configured according to the preset allocation strategy corresponding to the startup status. This solves the technical problem in the prior art where test traffic cannot be adjusted according to whether the cache is hit, resulting in a difference between the test results and the actual operating environment. This achieves the technical effect of improving test accuracy.

[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0089] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0090] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0091] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0092] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for configuring test resources, characterized in that, The method includes: Within a first preset time period, test request data is sent to the service to be tested, and test response parameters corresponding to each test request data are determined. The test request data includes multiple identical or similar test request data. Based on the test response parameters or the statistical characteristics of the test response parameters, the startup status of the service to be tested is determined. The startup status of the service to be tested is used to indicate whether there is cached data that the service to be tested can quickly access. Obtain the preset allocation strategy corresponding to the startup status of the service to be tested, and configure test resources according to the preset allocation strategy within a second preset time period.

2. The method according to claim 1, characterized in that, The startup status of the service to be tested includes cold start status and warm start status. The cold start status indicates that there is no cached data that the service to be tested can quickly access, while the warm start status indicates that there is cached data that the service to be tested can quickly access. The startup status of the service to be tested is determined in the following way: Determine the target response time interval into which at least one target test response parameter and / or the statistical characteristics of the target test response parameter fall, wherein the target response time interval includes a first response time interval, a second response time interval, and a third response time interval, wherein the lower limit of the third response time interval is not less than the upper limit of the first response time interval, and the upper limit of the third response time interval is not greater than the lower limit of the second response time interval; When the target response time interval is the first response time interval, the startup state of the service to be tested is the hot start state; When the target response time interval is the second response time interval, the startup state of the service to be tested is the cold start state; When the target response time interval is the third response time interval, the startup status of the service to be tested is determined based on the historical cache parameters and / or service operation parameters of the test request data.

3. The method according to claim 2, characterized in that, The first response time interval, the second response time interval, and the third response time interval corresponding to the service to be tested are constructed in the following manner: Obtain the actual response time or statistical characteristics of each actual response time corresponding to multiple actual request data, and the startup status of the service to be tested corresponding to each actual response time. The actual response time includes at least one first actual response time with a hot start status and at least one second actual response time with a cold start status. Sort the first true response time and the first statistical feature, and the second true response time and the second statistical feature, respectively. The upper limit of the third response time interval is calculated by interpolation based on the first true response time series after sorting or the first quantile of the first statistical feature. The lower limit of the third response time interval is calculated by interpolation based on the second true response time series after sorting or the second quantile of the second statistical feature.

4. The method according to claim 2, characterized in that, The historical cache parameters include the hash value of the cached data in the cache database and the cache lifecycle, and the service operation parameters include at least the server queue length or resource utilization rate. Based on the historical cache parameters and / or service operation parameters of the test request data, determine the startup status of the service to be tested, including: The hash value of the test request data is matched with the hash values ​​of each cached data in the cache database; When a hash value matching the hash value of the test request data exists in the cache database, the startup state of the service to be tested is the hot start state. When there is no matching hash value in the cache database, if the sending time parameter of the previous identical test request data and the currently sent test request data does not meet the auxiliary judgment condition, the startup state of the service to be tested is the cold start state. If the sending time parameter of the previous identical test request data and the currently sent test request data meets the auxiliary judgment condition, the startup status of the service to be tested is determined by auxiliary judgment based on the server queue length and / or the resource occupancy rate.

5. The method according to claim 4, characterized in that, The startup status of the service to be tested is determined by using the server queue length and / or the resource utilization rate as auxiliary criteria, including: When the server queue length is greater than a preset queue length threshold, and / or the resource utilization rate is greater than a preset resource utilization rate threshold, the startup state of the service to be tested is the hot start state. When the server queue length is less than or equal to a preset queue length threshold, and the resource utilization rate is less than or equal to a preset resource utilization rate threshold, the startup state of the service to be tested is the cold start state.

6. The method according to claim 4, characterized in that, Determine whether the auxiliary judgment conditions are met using the following methods: in, The probability density function represents the first true response time series or the first statistical characteristic. The probability density function represents the second true response time series or the second statistical characteristic, where F is the control factor and t refers to the response time or the statistical characteristic of the response time. TH is the judgment parameter corresponding to the auxiliary judgment condition. It is the difference in transmission time. The standard deviation represents the first true response time series or the first statistical characteristic. The standard deviation of the second true response time series or the second statistical characteristic is represented by C. and The maximum value; if TTL≤TH, the auxiliary judgment condition is not met; if TTL>TH, the auxiliary judgment condition is met. TTL represents the cache lifecycle corresponding to the previous request data that meets the similarity condition with the service under test.

7. The method according to claim 1, characterized in that, Configure test resources according to the preset allocation strategy within the second preset time period, including: Based on the preset allocation strategy, the second preset time period is divided into an initial stage, an intermediate stage, and an ending stage; The test traffic at each moment of the initial, intermediate, and final stages is configured using the stage allocation sub-strategies corresponding to the initial, intermediate, and final stages in the preset allocation strategy.

8. The method according to claim 7, characterized in that, The test traffic at each moment in the second preset time period is calculated using the following formula: in, This refers to the number of test request data at time t in the second preset time period when the startup state of the service to be tested is a hot start state; This refers to the flow control coefficient corresponding to the hot start state. , This refers to the maximum amount of test request data that the service can stably process during the hot start state. and All are positive numbers not greater than 1; This refers to the initial stage corresponding to the hot start state; This refers to the intermediate stage corresponding to the hot start state; This refers to the ending stage corresponding to the hot start state; This refers to the preset exponent. ; , For preset peak value, The minimum value is set to .

9. The method according to claim 7, characterized in that, The test traffic at each moment in the second preset time period is calculated using the following formula: in, This refers to the number of test request data at time t in the second preset time period when the startup state of the service to be tested is cold start state; This refers to the flow control coefficient corresponding to the cold start state. , This refers to the maximum amount of test request data that the service can stably process during the cold start state. and All are positive numbers not greater than 1; This refers to the initial stage corresponding to the cold start state; This refers to the intermediate stage corresponding to the cold start state; This refers to the ending stage corresponding to the cold start state.

10. A test resource configuration device, characterized in that, The device includes: The response parameter determination module sends test request data to the service to be tested within a first preset time period and determines the test response parameters corresponding to each test request data. The test request data includes multiple identical or similar test request data. The startup status determination module determines the startup status of the service to be tested based on the test response parameters or the statistical characteristics of the test response parameters. The startup status of the service to be tested is used to indicate whether there is cached data that the service to be tested can quickly access. The resource configuration module obtains the preset allocation strategy corresponding to the startup status of the service to be tested, and configures test resources according to the preset allocation strategy within a second preset time period.

Images