Data caching method, device and computer readable storage medium for recharge testing

Abstract

The embodiment of the present application relates to the technical field of data processing, and discloses a data caching method and device for recharge testing and a computer readable storage medium, the method comprising: determining a first cached data amount and a first cumulative difference according to historical download rate time series data; determining data amount deviation data according to the first cumulative difference and the first cached data amount; determining a target cached data amount according to the data amount deviation data and the first cached data amount; and performing data caching for recharge testing according to the target cached data amount. The technical scheme of the present application can dynamically adjust the cached data amount according to the real-time fluctuation of network download rate, ensure continuous supply of recharge testing data, and prevent testing interruption caused by insufficient caching.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a data caching method, apparatus, and computer-readable storage medium for backfeed testing. Background Technology

[0002] With the rapid development of intelligent driving technology, intelligent driving algorithms need to rely on a large amount of road-collected data for backfeed testing to verify the performance of perception, planning, and control algorithms in different scenarios. In actual testing, the amount of backfeed data is usually large, and the data transmission link is affected by factors such as network congestion, signal attenuation, and bandwidth jitter, so the download speed is not always stable.

[0003] Given the aforementioned technical background, traditional data rollback testing solutions typically rely on increasing network bandwidth and hard drive capacity to ensure data supply, or on setting a fixed cache size based on experience before initiating the rollback test. While these solutions can alleviate data shortages to some extent, they primarily depend on stacking hardware resources and are difficult to accurately match the actual caching needs under different network fluctuation conditions. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide a data caching method, apparatus and computer-readable storage medium for backfeeding tests, which solves the problem in the prior art that the required amount of cached data cannot be accurately determined in backfeeding tests.

[0005] According to one aspect of the present invention, a data caching method for backfeeding tests is provided, the method comprising: The first cached data volume and the first cumulative difference are determined based on historical download speed time-series data. The data volume deviation data is determined based on the first cumulative difference and the first cached data volume; The target cache data size is determined based on the data size deviation and the first cache data size. Data cache is used for backfilling tests based on the target cache data volume.

[0006] According to another aspect of the present invention, a data caching device for backfeed testing is provided, comprising: The data determination module is used to determine the first cached data volume and the first cumulative difference based on historical download rate time-series data; The deviation determination module is used to determine the data deviation data based on the first cumulative difference and the first cached data amount; The data determination module is also used to determine the target cache data amount based on the data amount deviation data and the first cache data amount; The data caching module is used for data caching tests based on the target cache data volume.

[0007] According to another aspect of the present invention, a data caching device for backfeeding test is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction that causes the processor to perform the data caching method described in the above-described backflow test.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein at least one executable instruction is stored in the storage medium, the executable instruction causing a data caching device / apparatus for backfeed testing to perform the operation of the data caching method for backfeed testing described above.

[0009] This invention relates to the estimation of cache demand resulting from historical download rate changes, correlates the deviation with the cumulative data gap corresponding to historical download rate time-series data, and determines the target cache data volume after correcting the first cache data volume based on the data volume deviation data. This ensures that the cache data volume before the start of the backfeed test no longer relies solely on fixed empirical values ​​or single historical statistical values, solving the problem of insufficient or excessive cache in existing backfeed tests due to the difficulty in accurately determining the cache data volume. Compared with existing technologies, this invention achieves higher consistency between the cache data volume and the data consumption gap under download rate fluctuations, reduces the probability of data interruption caused by insufficient cache, and reduces the invalid pre-store time and storage occupation caused by excessive cache, thereby improving the continuity of data supply and the effectiveness of cache resource allocation during the backfeed test.

[0010] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0011] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a first embodiment of the data caching method for backfeeding testing provided by the present invention is shown. Figure 2 A flowchart illustrating a second embodiment of the data caching method for backfeeding testing provided by the present invention is shown. Figure 3 A flowchart illustrating a third embodiment of the data caching method for backfeeding testing provided by the present invention is shown; Figure 4This paper illustrates a schematic diagram of the data caching process for a backfeed test, representing a third embodiment of the data caching method for backfeed testing provided by the present invention. Figure 5 A schematic diagram of the structure of a first embodiment of the data caching device for backfeed testing provided by the present invention is shown; Figure 6 A schematic diagram of an embodiment of the data caching device for backfeeding testing provided by the present invention is shown. Detailed Implementation

[0012] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0013] Figure 1 A flowchart of a first embodiment of the data caching method for backfeed testing according to the present invention is shown, the method being executed by a data caching device for backfeed testing. Figure 1 As shown, the method includes the following steps: Step 110: Determine the first cached data amount and the first cumulative difference based on historical download rate time-series data.

[0014] In this embodiment, historical download speed time-series data refers to a sequence of download speed values ​​collected continuously according to a preset sampling period during the backfeed test, arranged in chronological order. Historical download speed time-series data comprehensively records the changes in network download speed at different times, reflecting the historical fluctuation patterns of network transmission capacity.

[0015] In one implementation, the preset sampling period refers to a pre-defined time interval between two consecutive download rate acquisition operations. The value of the preset sampling period can be set adaptively, but it is necessary to balance the accuracy of data acquisition and the consumption of computing resources to ensure that the changing characteristics of network speed can be accurately captured.

[0016] In this embodiment, the constant consumption rate refers to the fixed rate at which test data is consumed during the intelligent driving data recovery test. The constant consumption rate is determined by the business characteristics of the recovery test and remains unchanged throughout the entire test process.

[0017] In this embodiment, the first cumulative difference refers to the cumulative difference calculated for each moment by simulating the data consumption process at a constant consumption rate within the download rate sequence within the sliding window. The difference between the total data consumption and the total data download at each moment is then taken as the maximum value among all cumulative differences. The first cumulative difference reflects the maximum gap between data consumption and data download when the network download rate is lower than the consumption rate in a recent period.

[0018] In one implementation, a sliding window refers to a dynamic data window maintained before the start of the backfeed test process to record the most recent download rate sampling values.

[0019] In this embodiment, the first cached data volume refers to the minimum cached data volume required for conducting the intelligent driving re-cracking test, calculated based on historical download rate time-series data using an improved Kingman approximation formula. The first cached data volume is used to address caching needs under stable network fluctuations and is the basic data volume that needs to be pre-cached before the re-cracking test begins.

[0020] For example, statistical feature parameters are extracted based on historical download rate time-series data, and the improved Kingman approximation formula is used to calculate the first cached data volume. Simultaneously, for the download rate sequence within the sliding window, the data consumption process is simulated at a constant consumption rate, and the cumulative difference at each time point is calculated, with the maximum value taken to obtain the first cumulative difference.

[0021] Step 120: Determine the data volume deviation data based on the first cumulative difference and the first cached data volume.

[0022] In this embodiment, data volume deviation data refers to a quantitative indicator used to quantify the difference between actual network fluctuations and theoretically calculated cache requirements. Data volume deviation data can intuitively reflect the degree of deviation between the current network fluctuation pattern and historical statistical patterns, providing a basis for determining whether to trigger cache correction.

[0023] In one implementation, before the backfeed test process begins, a sliding window is maintained, and the most recent preset number of download rate sample values ​​are recorded within the sliding window. Using the same cumulative difference calculation method as the historical download rate time series data, the download rate sequence within the sliding observation window is traversed and calculated to obtain the maximum value of the cumulative difference within the sliding observation window, and the maximum value of the cumulative difference within the sliding observation window is used as the first cumulative difference.

[0024] In this embodiment, the specific number of download rate sampling values ​​can be set according to requirements, and this embodiment does not impose any restrictions on this.

[0025] For example, the difference between the first accumulated difference and the first cached data amount is calculated, and the ratio of the difference between the first accumulated difference and the first cached data amount to the first cached data amount is used as the data amount deviation data. The formula for calculating the data amount deviation data is as follows:

[0026] In the formula, This is the first cumulative difference, which is the maximum value of the cumulative difference within the sliding window; This is the first cached data volume, which is the minimum cached data volume required to conduct intelligent driving data recovery tests.

[0027] Step 130: Determine the target cache data size based on the data size deviation data and the first cache data size.

[0028] In one implementation, the preset value refers to a pre-set threshold used to determine whether a significant change has occurred in the network fluctuation pattern. When the data deviation exceeds the preset value, an adaptive correction process for the cached data volume is triggered.

[0029] In this embodiment, the trust weight of online observation refers to the degree of bias towards the cumulative difference result obtained from real-time measurement when correcting the amount of cached data. The value of the trust weight of online observation determines the magnitude of the impact of real-time observation data on the final amount of cached data.

[0030] For example, when the data deviation exceeds a preset value, it is determined that the network fluctuation pattern has changed significantly. At this time, the first cached data volume is corrected to obtain the target cached data volume. For example, the preset value can be set to 15%. For example, the correction of the first cached data volume can be combined with the trust weight of online observation.

[0031] For example, when the data volume deviation does not exceed the preset value, the first cached data volume is directly determined as the target cached data volume.

[0032] Step 140: Perform data caching backflow testing based on the target cache data volume.

[0033] For example, before starting the online backfeed test process, test data is continuously downloaded until the cached data volume reaches or exceeds the target cached data volume, and then the data consumption process is started to carry out the backfeed test.

[0034] By offline analysis of historical network data to calculate basic caching requirements, combined with online sliding window monitoring of recent network fluctuations and calculation of real-time data gaps, and comparing the real-time gaps with theoretical caching requirements to obtain deviation data, the amount of cached data is dynamically adjusted based on the deviation. Finally, after pre-caching is completed according to the target amount of cached data, the test is started. This method can effectively resist network non-stability and sudden fluctuations, ensure the continuous and stable operation of the backfeed test, and avoid resource waste and startup delays caused by excessive caching.

[0035] Figure 2 A flowchart of another embodiment of the data caching method for backfeed testing according to the present invention is shown, which is performed by a data caching device for backfeed testing. Figure 2 As shown, the method includes the following steps: Step 210: Calculate the average download rate and the coefficient of variation of the squared download rate based on the historical download rate time-series data.

[0036] In this embodiment, the average download rate refers to the average of all download rate samples calculated based on historical download rate time-series data. The average download rate reflects the overall level of network transmission capacity over a historical period and is one of the core parameters for evaluating basic network performance.

[0037] In one implementation, the download rate squared coefficient of variation is the ratio of the average download rate variance to the square of the average download rate. The download rate squared coefficient of variation can eliminate the influence of the average rate magnitude on the assessment of fluctuation, accurately measuring the relative severity of download rate fluctuations.

[0038] For example, the average download rate is obtained by arithmetically averaging all sampled values ​​in the historical download rate time-series data. Simultaneously, the average rate variance is calculated based on the average download rate, and then the average rate variance is compared with the square of the average download rate to obtain the coefficient of variation of the squared download rate.

[0039] Step 220: Calculate the first cache data volume based on constant consumption rate, average download rate, historical download rate time-series data, and download rate squared variation coefficient.

[0040] For example, by combining constant consumption rate, average download rate, other statistical characteristic parameters extracted from historical download rate time-series data, and download rate squared variation coefficient, and substituting them into the improved Kingman approximation formula, the first cached data volume is obtained.

[0041] Step 230: Calculate the first cumulative difference based on the constant consumption rate, the preset sampling period, and the historical download rate time series data.

[0042] In one implementation, the preset sampling period refers to a pre-defined time interval between two consecutive download rate acquisition operations. The value of the preset sampling period needs to balance the accuracy of data acquisition and the consumption of computing resources to ensure that the changing characteristics of network speed can be accurately captured.

[0043] For example, simulating the data consumption process at a constant consumption rate, and combining a preset sampling period and historical download rate time-series data, the difference between the total data consumption and the total data download at each moment is calculated to obtain the cumulative difference. The maximum value of all cumulative differences is then taken throughout the entire time-series to obtain the first cumulative difference. The formula for calculating the cumulative difference is as follows:

[0044] In the formula, For a constant consumption rate; This is historical download speed time-series data; The preset sampling period is used. By traversing the entire time series, the historical maximum cumulative difference can be obtained. The formula for calculating the historical maximum cumulative difference is as follows:

[0045] In the formula, This represents the cumulative difference. The maximum cumulative difference within a sliding window can be calculated by examining the historical maximum cumulative difference, thus determining the maximum cumulative difference for the download rate sequence within that window.

[0046] By analyzing historical download speed time-series data offline, statistical parameters that reflect the overall network performance and fluctuation characteristics are extracted. At the same time, the largest data supply and demand gap in history is calculated, providing a comprehensive and reliable basis for the accurate calculation of subsequent cached data volume, ensuring that the theoretical cache demand can cover the extreme cases of historical network fluctuations.

[0047] Based on the foregoing embodiments, the data caching method for backfeeding tests provided in this embodiment of the invention calculates the average download rate and the coefficient of variation of the squared download rate based on historical download rate time-series data, which can be achieved through steps A1 to A3: Step A1: Calculate the average download speed based on historical download speed time-series data.

[0048] For example, the average download rate is obtained by summing all sampled values ​​in the historical download rate time-series data and then dividing the sum by the total number of sampled values. The formula for calculating the average download rate is as follows:

[0049] In the formula, This is historical download speed time-series data.

[0050] Step A2: Calculate the average rate variance based on the historical download rate and average download rate in the historical download rate time series data.

[0051] For example, calculate the difference between each historical download speed and the average download speed, square each difference, sum them up, and then divide the sum by the total number of samples to obtain the average speed variance. The formula for calculating the average speed variance is as follows:

[0052] In the formula, This is historical download speed time-series data; This represents the average download speed.

[0053] Step A3: Calculate the coefficient of variation of the squared download rate based on the average rate variance and the average download rate.

[0054] For example, dividing the average rate variance by the square of the average download rate yields the download rate squared coefficient of variation. The formula for calculating the download rate squared coefficient of variation is as follows:

[0055] In the formula, The average rate variance; This represents the average download speed.

[0056] By sequentially calculating the average download speed, average speed variance, and download speed squared coefficient of variation, the transmission characteristics of historical networks can be comprehensively characterized from three dimensions: overall level, absolute fluctuation amplitude, and relative fluctuation degree. This provides accurate input parameters for the subsequently improved Kingman approximation formula and ensures the reliability of the minimum buffer data size calculation results.

[0057] Based on the foregoing embodiments, the data caching method for backfeeding tests provided in this embodiment of the invention calculates the first cached data volume based on constant consumption rate, average download rate, historical download rate time-series data, and download rate squared variation coefficient, which can be achieved through steps A4 to A6: Step A4: Obtain the first empirical correction coefficient, the second empirical correction coefficient, the exponential decay factor, the arrival rate variation coefficient, the consumption data, the utilization rate, the steady state duration, and the decay rate coefficient.

[0058] In this embodiment, the first empirical correction coefficient refers to an empirical parameter used to correct the impact of non-stationary network traffic on cache requirements. The first empirical correction coefficient is obtained through offline historical data calibration and is used to adjust the contribution of arrival rate variability to the cache calculation results.

[0059] In one implementation, the second empirical correction coefficient is an empirical parameter used to correct the impact of network speed recovery on cache demand after a sudden rate drop. The second empirical correction coefficient is obtained through offline historical data calibration and is used to adjust the contribution of the exponential decay term to the cache calculation results.

[0060] In this embodiment, the exponential decay factor is a parameter used to describe the physical law of network speed gradually recovering after a sudden rate drop. The exponential decay factor can reflect the rate change trend of the network recovering from a congested state to a stable state.

[0061] In one implementation, the arrival rate variation coefficient is a parameter used to describe the additional contribution of network download rate variability to latency. The arrival rate variation coefficient can quantify the additional buffering requirements caused by non-stationary network traffic.

[0062] In this embodiment, consumed data refers to the amount of test data consumed per unit time during the intelligent driving data recovery test. Consumed data is directly related to a constant consumption rate and remains fixed throughout the entire test process.

[0063] In one implementation, utilization refers to the ratio of the network's average download speed to a constant consumption rate. Utilization reflects the degree to which network transmission capacity is saturated relative to the demand for test data consumption.

[0064] In this embodiment, the steady-state duration refers to the continuous duration during which the network is in a stable transmission state. The steady-state duration is used to calculate the degree of attenuation after a sudden rate drop.

[0065] In one implementation, the attenuation rate coefficient is a parameter used to adjust the attenuation rate of the exponential attenuation factor. The attenuation rate coefficient can accurately match the actual network speed recovery process based on the recovery characteristics of different network environments.

[0066] For example, the first empirical correction coefficient, the second empirical correction coefficient, the exponential decay factor, the arrival rate variation coefficient, and the decay rate coefficient are obtained from the parameter library after offline calibration. The utilization rate is calculated based on the constant consumption rate and the average download rate. The duration of the stable state is obtained based on historical network status records. The value of the consumption data is determined according to the constant consumption rate.

[0067] Step A5: Determine the first and second durations of the download rate decrease based on historical download rate time-series data.

[0068] In this embodiment, the first duration refers to the average duration of all rate decrease events in the historical download rate time-series data. The first duration can reflect the general duration characteristics of network rate decreases and is used to assess the duration of the impact of regular network fluctuations.

[0069] In one implementation, the second duration refers to the maximum duration of all rate drop events in the historical download rate time-series data. The second duration reflects the extreme duration of network rate drops and is used to cover the duration of the impact of the worst network fluctuations.

[0070] For example, iterate through historical download rate time-series data to identify all rate drop events where the download rate falls below the constant consumption rate. Calculate the duration of each rate drop event, and then average all durations to obtain the first duration. Finally, take the maximum value among all durations to obtain the second duration.

[0071] Step A6: Calculate the first cached data volume based on the constant consumption rate, consumption data, download rate squared variation coefficient, first empirical correction coefficient, second empirical correction coefficient, exponential decay factor, arrival rate variation coefficient, utilization rate, steady state duration, decay rate coefficient, first duration, second duration, and average download rate.

[0072] In one implementation, the improved Kingman approximation formula refers to a buffer calculation formula based on the improved Kingman approximation of the traditional G / G / 1 queuing model. The improved Kingman approximation formula adds non-stationary correction terms and burst decay terms, which can capture the non-stationary and bursty characteristics of network traffic.

[0073] For example, the constant consumption rate, consumed data, download rate squared coefficient of variation, first empirical correction coefficient, second empirical correction coefficient, exponential decay factor, arrival rate variation coefficient, utilization rate, steady-state duration, decay rate coefficient, first duration, second duration, and average download rate are all substituted into the improved Kingman approximation formula to perform numerical calculations to obtain the first cached data volume. The improved Kingman approximation formula is as follows:

[0074] In the formula, For a constant consumption rate; For utilization rate; The coefficient of variation of the squared download rate; For consuming data; This is the first empirical correction factor; The coefficient of variation for arrival rate; This is the first duration, i.e., the average duration of the rate decrease; This is the second empirical correction factor; It is an exponential decay factor; The duration for which the network remains in a stable state; This is the decay rate coefficient; This is the second duration, i.e., the maximum duration of the rate decrease.

[0075] In this embodiment, the first term of the improved Kingman approximation formula is the traditional Kingman approximation, which is the basic latency estimate for the G / G / 1 queuing mode under heavy traffic conditions. It is used to characterize the cache demand under steady fluctuations. For scenarios with a constant consumption rate C, the data consumption process is deterministic, i.e. At this time, the utilization rate .

[0076] The second term of the improved Kingman approximation is a non-stationary correction term, in which... The arrival rate variation coefficient is used to describe the additional contribution of arrival rate variability to the time delay. This is the first empirical correction coefficient.

[0077] The third term in the improved Kingman approximation is a sudden decay term, which introduces an exponential decay factor. This is used to describe the physical laws governing the gradual recovery of network speed after a sudden drop in speed. The duration for which the network remains in a stable state. The decay rate coefficient, This is the second empirical correction coefficient.

[0078] By comprehensively acquiring empirical correction parameters, network state parameters, and fluctuation characteristic parameters, and combining them with the duration of rate decline obtained from historical statistics, the improved Kingman approximation formula is used to calculate the first cached data size. This accurately characterizes the stability, non-stationarity, and burstiness of network traffic, ensuring that the calculated minimum cached data size can effectively resist various network fluctuations, while avoiding resource waste and startup delays caused by excessive caching.

[0079] Based on the foregoing embodiments, the data caching method for backfeeding tests provided in this embodiment of the invention further includes steps A7 to A9: Step A7: Obtain the regularization parameters.

[0080] In this embodiment, the regularization parameter refers to the adjustment parameter used to control the degree of overfitting during the parameter calibration process. The regularization parameter can balance the degree of fit of the calibration results to historical data and the generalization ability of the model, preventing the calibrated parameters from being overfitted to specific historical data.

[0081] For example, appropriate regularization parameter values ​​can be pre-set based on the scale and fluctuation characteristics of historical data for subsequent parameter calibration processes.

[0082] Step A8: Determine the second cumulative difference based on historical download speed time-series data.

[0083] In this embodiment, the second cumulative difference refers to the maximum historical cumulative difference calculated based on complete historical download rate time-series data, simulating the data consumption process at a constant consumption rate. The second cumulative difference reflects the largest data supply-demand gap that has occurred in all historical network fluctuations and serves as a reference for parameter calibration.

[0084] For example, the data consumption process is simulated at a constant consumption rate, and the cumulative difference at each moment is calculated by combining historical download rate time series data. The maximum value of all cumulative differences is obtained by traversing the entire time series.

[0085] Step A9: Calibrate the first empirical correction coefficient, the second empirical correction coefficient, and the attenuation rate coefficient based on the second cumulative difference and the regularization parameter.

[0086] For example, an objective function containing a second cumulative difference and a regularization parameter is constructed. By minimizing the value of the objective function, the calibrated first empirical correction coefficient, second empirical correction coefficient, and attenuation rate coefficient are obtained. The coefficient calibration calculation formula is as follows:

[0087] in For regularization terms, This is a regularization parameter to prevent overfitting; This is the first empirical correction factor; This is the decay rate coefficient; This is the second empirical correction factor; This is the second cumulative difference, which is the maximum value of the cumulative difference.

[0088] In one implementation, the first empirical correction coefficient, the second empirical correction coefficient, and the attenuation rate coefficient can be set as empirical initial values ​​based on general network environment characteristics to support the first intelligent driving recharge test.

[0089] Furthermore, before each subsequent backfeeding test is initiated, the first empirical correction coefficient, the second empirical correction coefficient, and the attenuation rate coefficient will be iteratively corrected based on the updated complete historical download rate time-series data. The corrected parameters will be directly applied to the calculation of the first cached data volume for this backfeeding test.

[0090] As the number of backfeed tests increases, the historical dataset will continuously accumulate more real data under different network fluctuation scenarios, the accuracy of parameter calibration will continue to improve, and the calculation results of the first cache data volume will become more and more in line with the actual network characteristics of the current deployment environment, realizing dynamic adaptive optimization of caching requirements.

[0091] By introducing regularization parameters to control the overfitting risk in the calibration process, and by calibrating the core empirical parameters in conjunction with the largest data supply-demand gap that has occurred in history, the improved Kingman approximation formula can better fit the fluctuation characteristics of the actual network, further improving the accuracy and generalization ability of the minimum cache data calculation results.

[0092] Figure 3 A flowchart of another embodiment of the data caching method for backfeed testing according to the present invention is shown, which is performed by a data caching device for backfeed testing. Figure 3 As shown, the method includes the following steps: Step 310: Obtain the deviation threshold.

[0093] In this embodiment, the deviation threshold refers to a pre-set numerical threshold used to determine whether the network fluctuation pattern has changed significantly. The value of the deviation threshold determines the sensitivity of triggering adaptive correction of the cached data volume, and needs to be set reasonably according to the stability requirements of the test service.

[0094] For example, a pre-set deviation threshold can be obtained from the preset parameter configuration and used for comparison and judgment of subsequent data deviation data.

[0095] Step 320: When the data volume deviation is greater than the deviation threshold, calculate the second cached data volume based on the first cumulative difference and the first cached data volume, and use the second cached data volume as the target cached data volume.

[0096] In this embodiment, the second cached data volume refers to the cached data volume corrected by combining the first cumulative difference observed in real time and the first cached data volume calculated theoretically when the network fluctuation pattern changes significantly. The second cached data volume can adapt to the latest network fluctuation characteristics and meet real-time caching requirements.

[0097] In one implementation, the trust weight of online observations refers to the degree of bias towards the first cumulative difference result obtained from real-time measurements when correcting the amount of cached data. The value of the trust weight of online observations determines the magnitude of the impact of real-time observation data on the final amount of cached data.

[0098] For example, when the data deviation exceeds the deviation threshold, the first cumulative difference and the first cached data volume are weighted and calculated in combination with the trust weight of online observation to obtain the second cached data volume, and the second cached data volume is determined as the target cached data volume.

[0099] Step 330: When the data volume deviation is less than or equal to the deviation threshold, the first cached data volume is used as the target cached data volume.

[0100] For example, when the data volume deviation is less than or equal to the deviation threshold, it is determined that the current network fluctuation pattern is not significantly different from the historical statistical pattern. Therefore, there is no need to correct the cached data volume, and the first cached data volume is directly determined as the target cached data volume.

[0101] By setting a preset deviation threshold to determine the degree of change in network fluctuation patterns, the first cached data volume calculated theoretically is used when the network fluctuation does not change significantly. When a significant change occurs, the second cached data volume is obtained by combining real-time observation data and used as the target cached data volume. This can ensure that cache requirements are covered while avoiding unnecessary cache adjustments, achieve dynamic adaptive optimization of cached data volume, and improve the stability and resource utilization efficiency of backfeed testing.

[0102] Based on the foregoing embodiments, in the data caching method for backfeeding tests provided by this embodiment of the invention, calculating the second cached data volume based on the first cumulative difference and the first cached data volume can be achieved through steps B1 to B2: Step B1: Obtain the current trust weight.

[0103] In this embodiment, the current trust weight refers to the degree of bias towards the first cumulative difference result obtained from real-time measurement when correcting the amount of cached data before the start of this backfeeding test. The value of the current trust weight is dynamically adjusted based on historical calibration results and the stability of the network environment to balance the contribution ratio of real-time observation data and historical statistical data.

[0104] For example, the current trust weight, which has been pre-set and historically calibrated, is obtained from the preset parameter configuration and used in the subsequent calculation of the second cache data volume.

[0105] Step B2: Calculate the second cached data volume based on the current trust weight, the first cumulative difference, and the first cached data volume.

[0106] For example, the first cumulative difference is weighted using the current trust weight, and the first cached data amount is weighted using a value minus the current trust weight. The two weighted results are then added together to obtain the second cached data amount. The formula for calculating the second cached data amount is as follows:

[0107] In the formula, This is the first cached data volume, which is the minimum cached data volume required to conduct intelligent driving data recovery testing; This is the first cumulative difference, which is the maximum value of the cumulative difference within the calculation window; This represents the current trust weight, i.e., the trust weight observed online.

[0108] By obtaining the current trust weight adapted to the current network environment, and combining it with the first cumulative difference observed in real time and the first cached data volume obtained from historical statistics to calculate the second cached data volume, the cache demand can be quickly adjusted to a reasonable level when the network fluctuation pattern changes significantly. This ensures that the backfeed test will not be interrupted due to cache exhaustion, and avoids resource waste and startup delay caused by excessive caching.

[0109] like Figure 4 As shown, Figure 4This paper illustrates a schematic diagram of the data caching process for backfeeding tests provided by this invention. The data caching method for backfeeding tests of this invention iterates through the historical download data time series, calculates the historical maximum difference based on the iterated historical download data time series, and extracts network fluctuation statistical characteristic parameters. The historical maximum difference characterizes the maximum data gap formed when the historical download rate is lower than the data consumption requirement of the backfeeding test. The network fluctuation statistical characteristic parameters characterize the average download rate, the degree of fluctuation, and the rate decline characteristics. Subsequently, an improved Kingman approximation formula is constructed, and the theoretical cache size is obtained using statistically measured parameters and correction coefficients. Theoretical cache size This is used to represent the initial cache requirement calculated based on historical network fluctuation characteristics and correction parameters; then, a backfeed test is initiated, a sliding flow window is constructed, and the maximum cumulative difference is calculated based on the sliding flow window. This maximum cumulative difference reflects the data volume gap formed under the current online download speed fluctuation; after obtaining the maximum cumulative difference, it is determined whether the deviation ε is greater than the threshold of 15%, where the deviation... Used to characterize the data gap obtained from current online computing and the theoretical cache size. The deviation between the values ​​is used to determine whether the current network fluctuation status has changed significantly compared to the historical statistical status. When the determination result is yes, adaptive correction is enabled, and the cache size is adjusted based on the current online data to obtain a cache value that is more suitable for the current network state of the backfeed test. When the determination result is no, the cache value is maintained, that is, the existing cache size is used for subsequent processing. After completing adaptive correction or maintaining the cache value, the backfeed test begins to consume data to perform backfeed, so that the backfeed test can read backfeed data from the cache according to the data consumption requirements.

[0110] Figure 5 A schematic diagram of an embodiment of the data caching device for backfeeding testing according to the present invention is shown. Figure 5 As shown, the device 500 includes: a data determination module 510, a deviation determination module 520, and a data buffer module 530.

[0111] The data determination module 510 is used to determine the first cached data volume and the first cumulative difference based on historical download rate time-series data.

[0112] The deviation determination module 520 is used to determine the data deviation data based on the first cumulative difference and the first cached data amount.

[0113] The data determination module 510 is also used to determine the target cache data volume based on the data volume deviation data and the first cache data volume.

[0114] The data caching module 530 is used for data caching for backfeeding tests based on the target cache data volume.

[0115] In an alternative embodiment, the data determination module 510 is also configured to calculate the average download rate and the coefficient of variation of the squared download rate based on historical download rate time-series data. The first cache data size is calculated based on constant consumption rate, average download rate, historical download rate time-series data, and download rate squared variation coefficient. The first cumulative difference is calculated based on a constant consumption rate, a preset sampling period, and historical download rate time-series data.

[0116] In an alternative embodiment, the data determination module 510 is also used to acquire a first empirical correction coefficient, a second empirical correction coefficient, an exponential decay factor, an arrival rate variation coefficient, consumed data, utilization rate, steady state duration, and decay rate coefficient. Determine the first and second durations of the download rate decrease based on historical download rate time-series data; The first cached data volume is calculated based on the constant consumption rate, consumption data, download rate squared variation coefficient, first empirical correction coefficient, second empirical correction coefficient, exponential decay factor, arrival rate variation coefficient, utilization rate, steady state duration, decay rate coefficient, first duration, second duration, and average download rate.

[0117] In an alternative embodiment, the data determination module 510 is also configured to calculate the average download rate based on historical download rate time-series data. Calculate the average rate variance based on the historical download rate and average download rate from the historical download rate time-series data; The coefficient of variation of the squared download rate is calculated based on the average rate variance and the average download rate.

[0118] In an alternative approach, the data determination module 510 is also used to obtain regularization parameters; The second cumulative difference is determined based on historical download speed time-series data; The first empirical correction coefficient, the second empirical correction coefficient, and the decay rate coefficient are calibrated based on the second cumulative difference and the regularization parameter.

[0119] In an alternative embodiment, the data determination module 510 is also used to obtain a deviation threshold; When the data volume deviation exceeds the deviation threshold, the second cached data volume is calculated based on the first cumulative difference and the first cached data volume, and the second cached data volume is used as the target cached data volume. When the data volume deviation is less than or equal to the deviation threshold, the first cached data volume is used as the target cached data volume.

[0120] In an alternative approach, the data determination module 510 is also used to obtain the current trust weight; The second cached data volume is calculated based on the current trust weight, the first cumulative difference, and the first cached data volume.

[0121] The data caching device for the backfeed test provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0122] Figure 6 The diagram shows a structural schematic of an embodiment of the data caching device for backfeeding testing according to the present invention. The specific embodiments of the present invention do not limit the specific implementation of the data caching device for backfeeding testing.

[0123] like Figure 6 As shown, the data caching device for the backfeed test may include: a processor 602, a communications interface 604, a memory 606, and a communications bus 608.

[0124] The processor 602, communication interface 604, and memory 606 communicate with each other via communication bus 608. Communication interface 604 is used to communicate with other network elements such as clients or other servers. The processor 602 executes program 610, specifically performing the relevant steps in the above-described embodiment of the data caching method for backfeeding testing.

[0125] Specifically, program 610 may include program code, which includes computer-executable instructions.

[0126] Processor 602 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The data cache device for the power-back test includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0127] Memory 606 is used to store program 610. Memory 606 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0128] Specifically, program 610 can be called by processor 602 to cause the data cache device for the backfeed test to execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0129] This invention provides a computer-readable storage medium storing at least one executable instruction. When the executable instruction is executed on a data caching device / app for backfeed testing, it causes the data caching device / app for backfeed testing to perform the data caching method for backfeed testing in any of the above method embodiments.

[0130] The executable instructions can be used to cause the data caching device / app for the backfeed test to execute the method provided in the above method embodiment. The implementation principle and technical effect are similar, and will not be described in detail here.

[0131] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Furthermore, the embodiments of this invention are not directed to any particular programming language.

[0132] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. Similarly, for the sake of brevity and to aid in understanding one or more aspects of the invention, in the description of exemplary embodiments of the invention above, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0133] Those skilled in the art will understand that the modules in the device of the embodiment can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiment can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.

[0134] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A data caching method for backfeeding tests, characterized in that, The method includes: The first cached data volume and the first cumulative difference are determined based on historical download speed time-series data. The data volume deviation data is determined based on the first cumulative difference and the first cached data volume; The target cache data volume is determined based on the data volume deviation data and the first cache data volume; The data cache is used for backfilling tests based on the target cache data volume.

2. The method according to claim 1, characterized in that, The step of determining the first cached data volume and the first cumulative difference based on historical download rate time-series data includes: Calculate the average download speed and the coefficient of variation of the squared download speed based on historical download speed time-series data; The first cache data volume is calculated based on the constant consumption rate, the average download rate, the historical download rate time-series data, and the download rate squared coefficient of variation. The first cumulative difference is calculated based on the constant consumption rate, the preset sampling period, and the historical download rate time series data.

3. The method according to claim 2, characterized in that, The calculation of the first cached data volume based on the constant consumption rate, the average download rate, the historical download rate time-series data, and the download rate squared coefficient of variation includes: Obtain the first empirical correction coefficient, the second empirical correction coefficient, the exponential decay factor, the arrival rate variation coefficient, the consumption data, the utilization rate, the steady state duration, and the decay rate coefficient. The first and second durations of the download rate decrease are determined based on the historical download rate time-series data. The first cached data amount is calculated based on the constant consumption rate, the consumption data, the download rate squared variation coefficient, the first empirical correction coefficient, the second empirical correction coefficient, the exponential decay factor, the arrival rate variation coefficient, the utilization rate, the steady state duration, the decay rate coefficient, the first duration, the second duration, and the average download rate.

4. The method according to claim 2, characterized in that, The calculation of the average download rate and the coefficient of variation of the squared download rate based on historical download rate time-series data includes: The average download speed is calculated based on historical download speed time-series data. Calculate the average rate variance based on the historical download rate and the average download rate in the historical download rate time-series data; The download rate squared variation coefficient is calculated based on the average rate variance and the average download rate.

5. The method according to claim 4, characterized in that, The method further includes: Get the regularization parameters; The second cumulative difference is determined based on the historical download rate time-series data; The first empirical correction coefficient, the second empirical correction coefficient, and the decay rate coefficient are calibrated based on the second cumulative difference and the regularization parameter.

6. The method according to claim 1, characterized in that, Determining the target cached data size based on the data size deviation data and the first cached data size includes: Obtain the deviation threshold; When the data deviation is greater than the deviation threshold, the second cached data volume is calculated based on the first cumulative difference and the first cached data volume, and the second cached data volume is used as the target cached data volume. When the data deviation is less than or equal to the deviation threshold, the first cached data amount is taken as the target cached data amount.

7. The method according to claim 6, characterized in that, The second cached data volume is calculated based on the first cumulative difference and the first cached data volume, including: Get the current trust weight; The second cached data volume is calculated based on the current trust weight, the first cumulative difference, and the first cached data volume.

8. A data caching device for a backfeed test, characterized in that, The device includes: The data determination module is used to determine the first cached data volume and the first cumulative difference based on historical download rate time-series data; The deviation determination module is used to determine the data volume deviation data based on the first cumulative difference and the first cached data volume; The data determination module is further configured to determine the target cache data amount based on the data amount deviation data and the first cache data amount; The data caching module is used for data caching for backfeeding tests based on the target cache data volume.

9. A data caching device for backfeeding testing, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation of the data caching method for the backfeed test as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on the data caching device / apparatus for the backflow test, causes the data caching device / apparatus for the backflow test to perform the operation of the data caching method for the backflow test as described in any one of claims 1-7.

Images