Method and system for adaptive setting of buffer data validity period, device, medium

Abstract

The application discloses a kind of self-adapting method and system of setting buffer data validity period, equipment, medium, the method first collects historical data and statistics each data's historical reference heat, to obtain the historical query distribution of data, then based on the historical query distribution of data and real-time query situation dynamically adjusts the validity period of current cache data, after the running length of current cache data reaches validity period, current buffer data is replaced.The self-adapting method of setting buffer data validity period of the application, by combining the historical query demand distribution of current cache data and real-time query distribution, the validity period of current cache data is dynamically adjusted adaptively, since the self-adapting setting of validity period is based on distribution characteristics, the adaptive setting result is more accurate, the cache hit rate is greatly improved, so as to improve the utilization efficiency of cache, and frequent cache replacement is not carried out, and I / O overhead is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of metrology data query optimization, in particular, to a method and system for adaptively setting the effective period of buffered data, an electronic device and a computer readable storage medium. BACKGROUND

[0002] With the development of the Internet of Things, the stable operation of the power grid is increasingly dependent on a large number of underlying sensing terminals. One type of large number of critical terminals is intelligent sensing terminal devices that collect voltage values and current values of power users. They usually collect power data of users at a fixed frequency and report to the metrology center for other users or systems to provide query and statistical analysis, thereby providing necessary support for power grid operation and management. Specifically, the metrology center associates the collected data with user profiles based on device IDs to obtain associated data, and finally stores the associated data in a database for further query and statistical analysis by other applications, such as statistical analysis of power load changes of residential users and power load changes of transformer areas.

[0003] However, due to the large number of collection terminals in the province, the data generated and the demand for retrieval queries are extremely large. Although placing data in the cache can effectively improve query speed, on the one hand, due to the limited cache space, it is impossible to place all data in the cache space, and on the other hand, if the content is blindly placed in the cache space, the hit rate may be low and the cache efficiency is low. Therefore, certain strategies need to be taken to determine the data stored in the cache. Common cache data replacement methods include the least recently used strategy, the first-in-first-out strategy, the least used strategy, and the random strategy. However, these methods often ignore the distribution of data query demand and cannot guarantee a high cache hit rate. In addition, frequent cache content replacement also brings a large I / O overhead. SUMMARY

[0004] The present application provides a method and system for adaptively setting the effective period of buffered data, an electronic device and a computer readable storage medium to solve the technical problem of low cache hit rate existing in the existing cache data replacement method.

[0005] According to one aspect of the present application, a method for adaptively setting the effective period of buffered data is provided, including the following content:

[0006] Collecting historical data and counting the historical reference heat of each data to obtain the historical query distribution of the data;

[0007] Based on the historical query distribution of the data and the real-time query situation, dynamically adjusting the effective period of the current cache data;

[0008] Replacing the current buffered data after the runtime of the current cache data reaches the effective period.

[0009] Further, the process of dynamically adjusting the validity period of the current cache data based on the historical query distribution and real-time query situation of the data includes the following contents:

[0010] Randomly initialize the validity period of the current cache data;

[0011] Calculate the historical reference value of each piece of current cache data within the initial validity period;

[0012] Obtain the real-time reference value of each piece of current cache data and subtract the historical reference value to calculate the real-time residual value of each piece of current cache data;

[0013] Update the real-time residual value of each piece of current cache data every preset time to obtain the mean and standard deviation of the real-time residual value of the current cache data;

[0014] Adaptively set the validity period of the current cache data based on the mean and standard deviation of the real-time residual value of the current cache data.

[0015] Further, the historical reference value of each piece of current cache data within the initial validity period is calculated based on the following formula:

[0016]

[0017] wherein, represents the historical reference value of the i-th data of the device, win represents the initial validity period, represents the running time of the current cache data, represents the historical reference value of the i-th data of the device, represents the historical reference value of the i-th data of the device, represents the historical reference value of the i-th data of the device, represents the historical reference value of the i-th data of the device, represents the historical reference value of the i-th data of the device,

[0018] Further, the real-time residual value of each piece of current cache data is calculated based on the following formula:

[0019]

[0020] wherein, represents the real-time residual value of the i-th data of the device, represents the real-time reference value of the i-th data of the device, represents the real-time reference value of the i-th data of the device, represents the real-time reference value of the i-th data of the device, represents the real-time reference value of the i-th data of the device, represents the real-time reference value of the i-th data of the device,

[0021] Further, the validity period of the current cache data is adaptively set based on the following formula:

[0022]

[0023] wherein win' represents the adaptive validity period, win max and win min respectively represent the maximum and minimum of the validity period of the cached data, win represents the initial validity period of the current cached data, and θ represents the validity period change scale, λ represents the weight, represents the mean function, represents the standard deviation function, and respectively represent the mean and standard deviation of the real-time residual value of the current cached data, and RT and σ respectively represent the mean and standard deviation of the historical reference value of the current cached data within the validity period.

[0024] Further, the process of replacing the current cached data comprises the following contents:

[0025] randomly initializing the validity period of the new data and resetting the running time of the cached data;

[0026] calculating the historical reference value of each piece of new data;

[0027] randomly selecting a plurality of pieces of new data from the plurality of pieces of new data based on the probability model to replace the current cached data.

[0028] Further, the process of randomly selecting a plurality of pieces of new data from the plurality of pieces of new data based on the probability model to replace the current cached data comprises the following contents:

[0029] calculating the probability value of each piece of new data to obtain a probability distribution;

[0030] mapping the probability value to a disc based on the roulette algorithm and dividing sectors based on the probability distribution;

[0031] generating a random number and determining the sector in which the random number falls through the binary search method, if the data corresponding to the sector is not in the hash table, the data is put into the cache space, otherwise, the step is repeated;

[0032] continuously repeating the above steps until the current cached data is replaced.

[0033] In addition, the present application also provides a system for adaptively setting the validity period of the cached data, comprising:

[0034] a data statistics module for collecting historical data and calculating the historical reference heat of each piece of data to obtain the historical query distribution of the data;

[0035] An adaptive setting module is configured to dynamically adjust the validity period of the current cache data based on the historical query distribution of the data and real-time query conditions.

[0036] A data replacement module is configured to replace the current cache data after the running time of the current cache data reaches the validity period.

[0037] In addition, the present application also provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the steps of the method described above by invoking the computer program stored in the memory.

[0038] In addition, the present application also provides a computer-readable storage medium configured to store a computer program for adaptively setting the validity period of cache data, wherein the computer program is configured to execute the steps of the method described above when running on a computer.

[0039] The present application has the following effects:

[0040] The method for adaptively setting the validity period of cache data according to the present application collects historical data and counts the historical reference heat of each piece of data, thereby obtaining the historical query distribution of the data, and then dynamically adjusts the validity period of the current cache data based on the historical query distribution of the data and real-time query conditions, and replaces the current cache data after the running time of the current cache data reaches the validity period. The method for adaptively setting the validity period of cache data according to the present application adaptively and dynamically adjusts the validity period of the current cache data by combining the historical query demand distribution and real-time query distribution of the current cache data. Since the validity period is adaptively set based on the distribution characteristics, the adaptively setting result is more accurate, the cache hit rate is greatly improved, the utilization efficiency of the cache is improved, and frequent cache replacement is avoided, thereby reducing the I / O overhead.

[0041] In addition, the system for adaptively setting the validity period of cache data according to the present application also has the above-mentioned advantages.

[0042] In addition to the above-mentioned objects, features and advantages, the present application has other objects, features and advantages. The present application will be further described below with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their descriptions are intended to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0044] Figure 1 is a flowchart of the method for adaptively setting the validity period of cache data according to the preferred embodiment of the present application.

[0045] Figure 2 is Figure 1 a sub-flowchart of step S2 in the method.

[0046] Figure 3 is Figure 1 a sub-flowchart of step S3 in the method.

[0047] Figure 4 is Figure 3 a sub-flowchart of step S33 in the method.

[0048] Figure 5 is a module structure diagram of the system for adaptively setting the valid period of buffered data according to another embodiment of the present application. DETAILED DESCRIPTION

[0049] The embodiments of the present application will be described in detail below with reference to the accompanying drawings, but the present application can be implemented in various different ways as defined and covered below.

[0050] It can be understood that, as shown in the preferred embodiment of the present application, a method for adaptively setting the valid period of buffered data is provided, which comprises the following contents: Figure 1

[0051] Step S1: collect historical data and count the historical reference heat of each piece of data to obtain the historical query distribution of the data;

[0052] Step S2: dynamically adjust the valid period of the current cached data based on the historical query distribution of the data and the real-time query situation;

[0053] Step S3: replace the current buffered data after the running time length of the current cached data reaches the valid period.

[0054] It can be understood that, in the method for adaptively setting the valid period of buffered data according to the present embodiment, the historical data is first collected and the historical reference heat of each piece of data is counted to obtain the historical query distribution of the data, then the valid period of the current cached data is dynamically adjusted based on the historical query distribution of the data and the real-time query situation, and the current buffered data is replaced after the running time length of the current cached data reaches the valid period. The method for adaptively setting the valid period of buffered data according to the present application adaptively and dynamically adjusts the valid period of the current cached data by combining the historical query demand distribution and the real-time query distribution of the current cached data, and the adaptively setting result is more accurate due to the adaptively setting of the valid period based on the distribution characteristics, which greatly improves the cache hit rate, thereby improving the utilization efficiency of the cache, and frequent cache replacement is not performed, which reduces the I / O overhead.

[0055] ​It is understood that in this invention, it is assumed that there are N intelligent sensing devices in the system, and each device collects K data points daily at a fixed frequency and reports them to the measurement center. This means that the measurement center will generate N*K related data points daily and store them in the database. Since measurement data has a certain time sensitivity, the older the data, the lower its value; that is, the older the data, the lower the probability of it being queried. It is assumed that the shared cache capacity of the measurement center can cache a maximum of H related data points at any given time. To maximize the utilization of the cache space, it is assumed that H related data points are always cached.

[0056] It is understood that in step S1, it is assumed that the cached data set in the current cache space is represented as D = {d1, d2, ..., d...} H}, where d h Indicates device The The data consists of several records. A cache hit is defined as a query where a record of data is stored in the cache and can be retrieved directly from the cache without querying the database. The reference popularity of a record at time t is defined as the number of times that record is queried at time t. Therefore, after collecting sufficient historical query data logs, the historical reference popularity of data k on device n at time t can be statistically determined. That is, the average number of times data k of device n is queried at time t, which represents the prediction of the number of times the data will be queried at time t.

[0057] Among them, such as Figure 2 As shown, in step S2, the process of dynamically adjusting the validity period of the current cached data based on the historical query distribution and real-time query situation includes the following:

[0058] Step S21: Randomly initialize the validity period of the current cached data;

[0059] Step S22: Calculate the historical reference value of each current cached data within its initial validity period;

[0060] Step S23: Obtain the real-time reference value of each current cached data and subtract it from the historical reference value to calculate the real-time remaining value of each current cached data.

[0061] Step S24: Update the real-time remaining value of each current cached data at preset intervals to obtain the mean and standard deviation of the real-time remaining value of the current cached data;

[0062] Step S25: Adaptively set the validity period of the current cached data based on the mean and standard deviation of the real-time remaining value of the current cached data.

[0063] Specifically, first, the validity period of the current cached data is randomly initialized to win, where win ∈ [win].min win max ], where win min and win max Let represent the minimum and maximum validity periods, respectively. This indicates the runtime of the current cached data. If ~t < win, it means that the cached data is within its validity period; otherwise, it indicates that the cached data has expired and needs to be replaced to ensure the efficiency of data query.

[0064] Then, the historical reference value of each currently cached data item within its initial validity period is calculated based on the following formula:

[0065]

[0066] in, Indicates device The The historical reference value of the data, win indicates the initial validity period. Indicates the runtime of the current cached data. Indicates device The Data at time Historical citation popularity.

[0067] Next, calculate the real-time reference value of the cached data. Let d h The real-time reference value is expressed as This value represents the total number of times the data has been queried in real-time after being placed in the cache, i.e., the number of times the data has been hit within the current cache's runtime. Afterwards, the historical reference value of each cached data item is... The real-time remaining value of each currently cached data item is calculated by subtracting its real-time reference value from its real-time reference value.

[0068] Specifically, the real-time remaining value of each currently cached data item is calculated based on the following formula:

[0069]

[0070] in, Indicates device The The real-time residual value of each piece of data Indicates device The The real-time reference value of the data. Obviously, the historical reference value represents the number of times the data can be queried within the valid period, which can be regarded as a prediction of the number of times the data is queried within the valid period based on massive historical data, and the real-time reference value represents the number of times the data is queried in real time. Therefore, the real-time residual value of a piece of data obtained by subtracting the two is the prediction of the number of times the data is queried in the subsequent valid period. The higher the value, the greater the probability that the data will be queried in the subsequent time, and vice versa, the lower the value, the smaller the probability that the data will be queried in the subsequent time.

[0071] Then, in order to reduce the amount of calculation and complexity, the characteristics of the data in the cache space are counted every interval of a preset time, that is, the real-time residual value of each cached data and the mean value of the real-time residual value are updated every interval of time and the standard deviation After updating these characteristics, the cache valid period is adaptively adjusted.

[0072] Obviously, the greater the mean value , the greater the overall residual value of the cached data, that is, the data query demand may be concentrated in the subsequent valid period, and at this time the cache valid period should be extended; however, the greater the standard deviation , the greater the volatility of the data, that is, the residual value of the data is unevenly distributed, a small amount of data occupies most of the residual value, and most of the data has a low residual value, so the utilization rate of the cache space is low, and at this time the cache valid period needs to be shortened to replace the cached data in time and improve the utilization rate of the cache space and the efficiency of the query. Therefore, the application designs an algorithm for adaptively setting the cache valid period, specifically, the valid period of the current cached data is adaptively set based on the following formula:

[0073] win' = min{win max , max(win min , win x (1 + theta))

[0074] wherein win' represents the adaptive valid period, win max and win min represent the maximum and minimum values of the cache data valid period respectively, win represents the initial valid period of the current cached data, theta represents the valid period change scale, lambda represents the weight, the mean value function, the standard deviation function, and respectively, and RT and σ represent the mean and standard deviation of the historical reference value of the current cache data within the validity period, respectively. It can be understood that the weight λ represents the influence of the mean and the standard deviation on the scale of the validity period. If the mean is greater than the standard deviation, the mean is more distinctive, and thus it has a greater influence on the scale of the validity period. Conversely, the standard deviation has a greater influence on the scale of the validity period. In addition, the definitions of the mean function f1(x) and the standard deviation function f2(x, σ) are as follows:

[0075]

[0076]

[0077] where maxυ represents the maximum value of the scale of the validity period, that is, θ ∈ [-maxυ, maxυ], w1 represents a slope, w2 represents a weighting coefficient, both of which are constants, and w1 > 0 and w2 > 1, thre1 represents an offset constant, and ε represents a very small normal number close to 0,

[0078] It can be understood that if only the mean of the real-time residual value is considered, the overall query trend of the data cannot be accurately described. If the real-time hit rate of the data is low for a long time, the residual value of the data will remain at a high level. Obviously, even if the real-time residual value of the data is high, the low hit rate means that the data is rarely used in actual queries. Thus, although the data has a high value within the residual validity period, it may not be able to fully play its potential role. Therefore, in the present application, in order to more accurately evaluate the value of the data, the mean function f1(x) considers the real-time residual value and the real-time cache hit rate, and adaptively adjusts the cache validity period from a global perspective. If the cache value and the cache hit rate are both high, the function value is larger, that is, the cache validity period is extended. Conversely, the value of any one of them becomes smaller, which has a negative impact, and thus the function value becomes smaller. For the standard deviation function f2(x, σ), the closer the scale of the change of the standard deviation σ close to the historical reference value is to 0, and the smaller the standard deviation of the residual value is in the overall trend, the larger the value of the scale of the change of the validity period is. Conversely, the larger the standard deviation of the residual value is, the smaller the value of the standard deviation function is, which realizes the adaptive cache validity period from a local perspective. Finally, the result of the weighted function f1(x) and the function f2(x, σ) realizes the adaptive setting of the cache validity period, which balances the global perspective and the local perspective, and the adaptive result of the validity period is more reasonable and accurate.

[0079] Based on the above process, the cache validity period is continuously iteratively updated. When the running time of the cache data If the running time of the cache data exceeds the adaptive cache validity period win', the next step is entered, otherwise the current step is continued. In addition, as the running time is increasing all the time, although the validity period is adaptively adjusted every certain time, as long as the running time exceeds the adaptive validity period win' before or after the adaptive adjustment, the next step is entered.

[0080] It can be understood that, as the query demand of data is changing all the time, the query distribution at different time is different, which means that a static cache strategy cannot achieve a better cache hit rate. At present, the validity period of the cache is manually set by the developer in the configuration, but this static setting may not adapt to the changes of different resources and access modes. For example, if the validity period is set too short, it may cause frequent replacement of data in the cache space, although the hit rate of the query may be high at this time, but it will also cause a large I / O overhead, resulting in low overall efficiency; if the validity period is set too long, it will cause the data replacement not timely, resulting in a low cache hit rate. Therefore, the application dynamically adjusts the validity period of the cache data based on the historical query distribution and real-time query of the data, and comprehensively considers the real-time hit rate and real-time residual value distribution characteristics of the cache data for adaptive adjustment, so that the result of the adaptive adjustment is more accurate, which is beneficial to improve the utilization efficiency of the cache.

[0081] It can be understood that, in the step S3, when the running time of the cache data exceeds the validity period win', the current cache data in the buffer space is replaced. As shown in the figure, the process of replacing the current buffer data includes the following contents: Figure 3

[0082] Step S31: randomly initializing the validity period of the new data and resetting the running time of the cache data;

[0083] Step S32: calculating the historical reference value of each new data;

[0084] Step S33: randomly selecting a plurality of new data from a plurality of new data based on a probability model to replace the current cache data.

[0085] Specifically, as new data is generated at any time, a plurality of new data which are not cached to the cache space will be generated within the running time of the current cache data, and these new data can be represented as a set M={b1, b2,..., b M}, where b m represents the i-th data of the device . The validity period of the new data is randomly initialized as win∈[win , win']. ​​​​min win max [and reset the runtime of cached data]

[0086] Then, calculate the historical reference value of each new data item in set M within the initial validity period win, specifically based on data b. m Historical reference popularity is used to calculate its historical reference value within the initial validity period win. m , can be represented as:

[0087]

[0088] Because cached data is real-time, new data is generally far more valuable than older data stored in the cache. Therefore, it's necessary to consider replacing cached data when its expiration date arrives. To speed up the replacement process, and under the aforementioned assumptions, all old data in the cache will be discarded, freeing up H data slots in the cache space. Historical reference popularity, to some extent, reflects the degree to which data is used in subsequent queries. However, high historical reference popularity does not necessarily indicate high real-time reference popularity. Therefore, this invention uses a probabilistic model to randomly select H data items from set M to replace the current cached data in the cache space.

[0089] Among them, such as Figure 4 As shown, the process of randomly selecting multiple new data entries from a pool of new data entries based on a probabilistic model to completely replace the current cached data includes the following:

[0090] Step S331: Calculate the probability value of each new data point being selected to obtain the probability distribution;

[0091] Step S332: Map the probability values ​​onto a disk based on the roulette wheel algorithm, and divide the sectors based on the probability distribution;

[0092] Step S333: Generate a random number and use a binary search method to determine which sector the random number falls into. If the data corresponding to the sector is not in the hash table, put the data into the cache space; otherwise, repeat this step.

[0093] Step S334: Repeat the above steps until all the current cached data has been replaced.

[0094] Specifically, we first calculate the probability of selecting M new data points, with a time complexity of O(M), using the following formula:

[0095]

[0096] Among them, P select (υ m ) represents data bm The selected probability.

[0097] Then, the probability value is mapped to a disc based on the roulette algorithm, the sectors are divided based on the probability distribution, and the corresponding data is selected to be put into the cache space according to which sector the random number falls in. For example, the boundary set of sector division on the disc is obtained by simplifying the probability accumulation, and the length of the boundary set is M+1, that is, Obviously, the size of the i-th sector is P S [i]-P S [i-1]=P select (υ i ). Obviously, the time complexity of this step is also O(M).

[0098] Next, a random number is generated, and the binary search method is used to determine in which sector the random number falls. If the data corresponding to the sector is not in the hash table, the data is put into the cache space and added to the hash table. Otherwise, repeat this step. It can be understood that, in order to avoid caching the same data, the application maintains a hash table with a length of H. If the randomly obtained data exists in the hash table, it will not be put into the cache space. Therefore, the time complexity of this step is O(log2M).

[0099] Finally, step S333 is repeated until the number of data put into the buffer space reaches H. At this time, the length of the hash table also reaches H. Therefore, the total time complexity is O(Hlog2M). Since M is generally much greater than H, that is, M>Hlog2M, the overall time complexity of the cache replacement algorithm of the application is O(M), and the space complexity is also O(M).

[0100] It can be understood that, by using the probability model to randomly select data to be put into the cache, the application can obtain a better cache hit rate with a lower time complexity, and balance the time efficiency and cache efficiency. Compared with real-time high-frequency replacement, the cache replacement method of the application can reduce the frequency of cache replacement, realize batch replacement, improve the cache data replacement efficiency, and reduce the I / O cost. In addition, since the cache validity period is adaptively adjusted, the cache hit rate can be effectively guaranteed.

[0101] In addition, as shown in Figure 5 Another embodiment of the application also provides a system for adaptively setting the validity period of buffer data, which preferably uses the method described above, comprising:

[0102] A data statistics module is configured to collect historical data and statistically analyze the historical reference heat of each data to obtain the historical query distribution of the data.

[0103] An adaptive setting module is configured to dynamically adjust the validity period of the current cache data based on the historical query distribution of the data and the real-time query condition;

[0104] A data replacement module is configured to replace the current cache data after the running time of the current cache data reaches the validity period.

[0105] It can be understood that the system for adaptively setting the validity period of the cache data according to the embodiment collects historical data and counts the historical reference heat of each piece of data, thereby obtaining the historical query distribution of the data, and then dynamically adjusts the validity period of the current cache data based on the historical query distribution of the data and the real-time query condition, and replaces the current cache data after the running time of the current cache data reaches the validity period. The system for adaptively setting the validity period of the cache data according to the embodiment adaptively and dynamically adjusts the validity period of the current cache data by combining the historical query requirement distribution and the real-time query distribution of the current cache data. Since the validity period is adaptively set based on the distribution characteristics, the adaptive setting result is more accurate, the cache hit rate is greatly improved, the utilization efficiency of the cache is improved, and frequent cache replacement is avoided, thereby reducing the I / O overhead.

[0106] In addition, another embodiment of the present application further provides an electronic device including a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the steps of the method described above by invoking the computer program stored in the memory.

[0107] In addition, another embodiment of the present application further provides a computer readable storage medium for storing a computer program for adaptively setting the validity period of the cache data, wherein the computer program is configured to execute the steps of the method described above when running on a computer.

[0108] The computer readable storage medium can be a machine-readable storage medium, e.g., any tangible medium of expression that can be used to express the instructions. The instructions can be executed using one or more processors. The instructions can be encoded on any computer readable medium for execution by a processor or processors. Examples of computer readable media include a floppy disk, flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a Compact Disc Read Only Memory (CD-ROM), any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), a FLASH- EPROM, any other memory chip or cartridge, or any other medium from which a computer can read. Instructions can further be transmitted or received over a transmission medium. The term "transmission medium" means a tangible or non-tangible medium that can be used to store, encode or carry the instructions for execution by a machine, and includes any medium that is used for like purpose. Transmission media includes a baseband transmission, a propagation curve, and a carrier wave transmitting signals. Examples of transmission media include a coaxial cable, a copper wire, and fiber optics, including the wires that comprise a bus that carries digital data signals.

[0109] The specific embodiments of the present application have been shown by way of example in the above-described embodiments. It should be noted that various substitutions and alterations are possible in the application without departing from the spirit and scope of the application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application should be included in the scope of the protection of the present application.

[0110] Those skilled in the art will appreciate that embodiments of the present application can be supplied as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) having computer usable program code embodied thereon. The aspects of the embodiments of the present application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript, etc.

[0111] The present application is described in terms of flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions of the flowcharts and / or the block diagrams. Figure 1one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart

[0112] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart Figure 1 one or more processes and / or blocks Figure 1 the functions specified in the flowchart.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the flowchart Figure 1 one or more processes and / or blocks Figure 1 the steps of the functions specified in the flowchart.

[0114] Although preferred embodiments of the application have been described herein, those skilled in the art will readily devise many additional variations of these preferred embodiments that will fall within the scope of the present application. Accordingly, the appended claims are intended to encompass all variations of the preferred embodiments that fall within the scope of the present application.

[0115] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described herein.

Claims

1. A method for adaptively setting a buffer data validity period, characterized by, The method comprises the following steps: collecting historical data and counting historical reference heat of each piece of data to obtain historical query distribution of the data; dynamically adjusting effective period of current cache data based on historical query distribution of the data and real-time query situation; replacing current cache data after running time of the current cache data reaches the effective period; the process of dynamically adjusting the effective period of the current cache data based on the historical query distribution of the data and the real-time query situation comprises the following steps: randomly initializing the effective period of the current cache data; calculating historical reference value of each piece of current cache data within the initial effective period; obtaining real-time reference value of each piece of current cache data and calculating real-time residual value of each piece of current cache data by subtracting the historical reference value from the real-time reference value; updating the real-time residual value of each piece of current cache data every preset time to obtain mean value and standard deviation of the real-time residual value of the current cache data; adaptively setting the effective period of the current cache data based on the mean value and the standard deviation of the real-time residual value of the current cache data; adaptively setting the effective period of the current cache data based on the following formula: win' = min{win max ,max(win min ,win x (1 + 0))} where win' denotes the adaptive validity period, win max and win min denote the maximum and minimum of the validity period of cached data, respectively, win denotes the initial validity period of the current cached data, and θ denotes the validity period variation scale, λ denotes the weight, and denote the mean and standard deviation of the real-time residual value of the current cached data, respectively, and RT and σ denote the mean and standard deviation of the historical reference value of the current cached data within the validity period, respectively, denotes the mean function, denotes the standard deviation function, where, maxυ denotes the maximum of the validity period variation scale, i.e. θ ∈ [-maxυ, maxυ], w1 denotes the slope, w2 denotes the weighting coefficient, both of which are constants, and w1 > 0, w2 > 1, thre1 denotes the offset constant, and ε denotes a very small positive number close to 0, 2. The method for adaptively setting a buffer data validity period according to claim 1, wherein, calculating the historical reference value of each piece of current cache data within the initial effective period based on the following formula: wherein, representing the device the historical reference value of the first piece of data, win represents the initial validity period, represents the runtime length of the current cache data, representing the device the historical reference heat of the first piece of data at time .

3. The method for adaptively setting a buffer data validity period according to claim 2, wherein, calculating the real-time residual value of each piece of current cache data based on the following formula: wherein, representing the device the first data of the real-time residual value, representing the device the first data of the real-time reference value.

4. The method for adaptively setting a buffer data validity period of claim 1, wherein, the process of replacing the current cache data comprises the following steps: randomly initializing the effective period of new data and resetting running time of cache data; calculating historical reference value of each piece of new data; randomly selecting a plurality of pieces of new data from a plurality of pieces of new data based on a probability model to replace the current cache data.

5. The method for adaptively setting a buffer data validity period according to claim 4, wherein, the process of randomly selecting a plurality of pieces of new data from a plurality of pieces of new data based on a probability model to replace the current cache data comprises the following steps: calculating probability value of each piece of new data to obtain probability distribution; mapping the probability value to a disc based on a roulette algorithm and dividing sectors based on the probability distribution; generating a random number and determining the sector in which the random number falls by using a binary search method, if the data corresponding to the sector is not in a hash table, the data is put into a cache space, otherwise, the step is repeated; continuously repeating the above steps until the current cache data is replaced.

6. A system for adaptively setting a validity period of buffered data, employing the method for adaptively setting a validity period of buffered data according to any one of claims 1 to 5, characterized in that The method comprises the following steps: a data counting module for collecting historical data and counting historical reference heat of each piece of data to obtain historical query distribution of the data; an adaptive setting module for dynamically adjusting effective period of current cache data based on historical query distribution of the data and real-time query situation; a data replacement module for replacing current cache data after running time of the current cache data reaches the effective period.

7. An electronic device, comprising: The computer program runs on a computer to execute the steps of the method of any one of claims 1-5.

8. A computer-readable storage medium storing a computer program for adaptively setting a data validity period of a buffer, characterized by The computer program runs on a computer to execute the steps of the method of any one of claims 1-5.

Images