Caching method and system with adaptive adjustment based on LRU differentiation

Abstract

Provided in the embodiments of the present invention are a caching method and system with adaptive adjustment based on LRU differentiation. The method comprises: configuring a basic admission threshold value in a cache queue, performing iterative adjustment on the basic admission threshold value, calculating an adjusted hit rate after the adjustment, and determining a mapping relationship on the basis of a standard admission threshold value and the adjusted hit rate after the adjustment; after target data is received, determining a target adjustment threshold value on the basis of the mapping relationship, and screening the target data by means of the target adjustment threshold value, so as to determine target cache data; determining a selection result and a priority ranking result of the target data, so as to obtain preferential cache data; and acquiring metadata from the preferential cache data for screening, determining final cache data, and putting the final cache data into the cache queue. In this way, executable codes can be generally used to complete an automatic test, thereby reducing the workload and repeatability of automation; and adaptive adjustment is flexibly performed on a cache queue according to a differentiated scenario, thereby meeting the user requirements.

Description

A cache method and system based on adaptive adjustment of LRU differentiation Technical Field

[0001] The present invention relates to the technical field of data caching for data storage, and in particular to a caching method and system based on adaptive adjustment of LRU differentiation. Background Art

[0002] The most commonly used cache algorithm, LRU (Least Recently User), eliminates data based on its access history. Its core principle is that if data has been accessed recently, it is likely to be accessed again in the future. Currently, LRU and other eviction algorithms are used to cache all data accesses, keeping the most popular data in the cache and evicting the oldest, unused data.

[0003] Currently, when there is hot data in the cache queue, LRU is very efficient. However, occasional and periodic batch data operations will cause the LRU hit rate to drop sharply, and the cache pollution is relatively serious, which will lead to a relatively high degree of performance degradation. To address this situation, admission policies are sometimes added to the cache queue, but the current LRU policy is relatively simple, without flexible configuration functions, and cannot adaptively adjust the cache admission policy. It also lacks optimization for specific scenarios and more fine-grained data processing.

[0004] Summary of the Invention

[0005] In view of the problems existing in the prior art, the embodiments of the present invention provide a cache method and system for adaptive adjustment based on LRU differentiation.

[0006] An embodiment of the present invention provides a cache method for adaptive adjustment based on LRU differentiation, including:

[0007] A basic admission threshold is configured in the cache queue, and after filtering input data using the basic admission threshold to determine cached data, a data hit rate of the cached data when a data request is received is calculated, and the basic admission threshold is iteratively adjusted to calculate an adjusted hit rate after adjustment, and a mapping relationship is determined based on the adjusted standard admission threshold and the adjusted hit rate;

[0008] Repeating the iterative adjustment process, and adjusting the data attributes of the input data during the iterative adjustment process, combining the Markov model, and statistically determining a mapping relationship between the standard admission threshold and the adjusted hit rate under different data attributes;

[0009] After receiving the target data, obtaining the data attributes of the target data, and based on the target mapping relationship corresponding to the data attributes of the target data, searching for the target adjustment threshold corresponding to the maximum adjustment hit rate in the target mapping relationship, and screening the target data by the target adjustment threshold to determine the target cache data;

[0010] Obtaining a data classification and a data scenario corresponding to the target data, and determining a selection result and a priority sorting result of the target data according to the data classification and the data scenario, performing data selection on the target cache data based on the selection result, and prioritizing the target cache data after the data selection according to the priority sorting result, and determining the priority cache data after the priority sorting;

[0011] The metadata in the priority cache data is obtained, the throughput corresponding to the metadata is counted, the metadata in the priority cache data is filtered based on the throughput, the filtered final cache data is determined, and the final cache data is placed in a cache queue.

[0012] In one embodiment, the method further comprises:

[0013] Acquire corresponding historical data based on the data attributes of the target data, and detect the historical data type and the corresponding historical data scenario corresponding to the data request frequency of the historical data;

[0014] Obtaining a call sequence record of data in the historical data;

[0015] The selection result of the target data is determined based on the historical data type and the historical data scenario, and the priority sorting result is determined by the calling sequence record.

[0016] In one embodiment, the method further comprises:

[0017] Obtaining a request hit rate of the data request hitting cached data in the input data, and determining an average request rate corresponding to the data request;

[0018] A data hit rate of the cached data is calculated based on the request hit rate and the average request rate.

[0019] In one embodiment, the method further comprises:

[0020] The metadata is tracked through the index nodes, and the managed XFS file system finds the corresponding files of the index nodes on the disk, and determines the corresponding throughput based on the number of the index nodes.

[0021] In one embodiment, the method further comprises:

[0022] When a target data request for the cache queue is received, if the number of requests for a single data in the target data request is less than a preset threshold, the data order of the final cached data in the cache queue is not adjusted.

[0023] In one embodiment, the calculation formula of the data hit rate includes:

[0024] Where OHR(c) is the ratio of data hits to data requests, i is the object of data request, r i is the up-shift rate of object i in the time interval between the data request in the previous stage and the next data request.

[0025] An embodiment of the present invention provides a cache system with adaptive adjustment based on LRU differentiation, including:

[0026] an iterative adjustment module, configured to configure a basic admission threshold in a cache queue, and after filtering input data using the basic admission threshold to determine cached data, calculate a data hit rate of the cached data when a data request is received, iteratively adjust the basic admission threshold, calculate an adjusted hit rate after adjustment, and determine a mapping relationship based on the adjusted standard admission threshold and the adjusted hit rate;

[0027] A mapping module is configured to repeat the iterative adjustment process, adjust the data attributes of the input data during the iterative adjustment process, and determine a mapping relationship between the standard admission threshold and the adjusted hit rate under different data attributes by combining a Markov model;

[0028] a screening module configured to, upon receiving target data, obtain data attributes of the target data, and based on a target mapping relationship corresponding to the data attributes of the target data, search for a target adjustment threshold corresponding to a maximum adjustment hit rate in the target mapping relationship, and screen the target data using the target adjustment threshold to determine target cache data;

[0029] a priority module, configured to obtain a data classification and a data scenario corresponding to the target data, and determine a selection result and a priority sorting result of the target data according to the data classification and the data scenario, and perform data selection on the target cache data based on the selection result, and perform priority sorting on the target cache data after the data selection according to the priority sorting result, and determine the priority cache data after the priority sorting;

[0030] The metadata module is used to obtain the metadata in the priority cache data, count the throughput corresponding to the metadata, filter the metadata in the priority cache data based on the throughput, determine the final cache data after filtering, and put the final cache data into the cache queue.

[0031] In one embodiment, the system further comprises:

[0032] a detection module, configured to obtain corresponding historical data based on data attributes of the target data, and detect a historical data type and a corresponding historical data scenario corresponding to a data request frequency of the historical data;

[0033] An acquisition module, used for acquiring a call sequence record of data in the historical data;

[0034] The determination module is used to determine the selection result of the target data based on the historical data type and the historical data scenario, and the calling sequence record determines the priority sorting result.

[0035] An embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned cache method for adaptive adjustment based on LRU differentiation are implemented.

[0036] An embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned cache method for adaptive adjustment based on LRU differentiation are implemented.

[0037] The embodiment of the present invention provides a cache method and system based on LRU differentiated adaptive adjustment, which configures a basic access threshold in the cache queue, and after filtering the input data through the basic access threshold to determine the cache data, calculates the data hit rate of the cache data when a data request is received, and iteratively adjusts the basic access threshold, calculates the adjusted hit rate, and determines a mapping relationship based on the adjusted standard access threshold and the adjusted hit rate; repeats the iterative adjustment process, and adjusts the data attributes of the input data during the iterative adjustment process, combines the Markov model, and determines a mapping relationship between the standard access threshold and the adjusted hit rate under different data attributes; when the target data is received, obtains the data attributes of the target data, and corresponds to the target data based on the data attributes of the target data. The target mapping relationship is searched for the target adjustment threshold corresponding to the maximum adjustment hit rate in the target mapping relationship, and the target data is filtered through the target adjustment threshold to determine the target cache data; the data classification and data scenario corresponding to the target data are obtained, and the selection result and priority sorting result of the target data are determined based on the data classification and data scenario, and the target cache data is selected based on the selection result, and the target cache data after data selection is prioritized according to the priority sorting result to determine the priority cache data after priority sorting; the metadata in the priority cache data is obtained, the throughput corresponding to the metadata is counted, and the metadata in the priority cache data is filtered based on the throughput to determine the final cache data after filtering, and the final cache data is queued into the cache queue. In this way, the optimal cache threshold for input data can be automatically determined, the object hit rate can be optimized, and the application execution speed can be improved. Through priority sorting and selective allocation, storage elements that can improve performance can be identified and cached. For metadata classification cache, the performance of specific scenarios can be improved, so that the cache queue can be flexibly and adaptively adjusted according to differentiated scenarios to meet user needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] FIG1 is a flow chart of a caching method based on LRU differentiation and adaptive adjustment according to an embodiment of the present invention;

[0040] FIG2 is a structural diagram of a cache system based on LRU differentiation and adaptive adjustment according to an embodiment of the present invention;

[0041] FIG3 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0043] FIG1 is a flow chart of a caching method for adaptive adjustment based on LRU differentiation provided by an embodiment of the present invention. As shown in FIG1 , an embodiment of the present invention provides a caching method for adaptive adjustment based on LRU differentiation:

[0044] Step S101: configure a basic access threshold in the cache queue, and after filtering the input data through the basic access threshold to determine the cached data, calculate the data hit rate of the cached data when a data request is received, and iteratively adjust the basic access threshold, calculate the adjusted hit rate, and determine the mapping relationship based on the adjusted standard access threshold and the adjusted hit rate.

[0045] Specifically, a basic access threshold is configured in the cache queue, wherein the basic access threshold is an access threshold based on the object size, which can be an experience threshold set in the current prior art and can be preset by relevant staff based on their work experience in this field. The basic access threshold can filter the cached data in the cache queue. When the object size of the cached data is larger than the basic access threshold, in order to ensure the number of data in the cache queue, the corresponding cached data will not be saved to the cache queue, and only cached data smaller than the basic access threshold will be saved to the cache queue. After the input data is filtered and the cached data is determined by the basic access threshold, based on the current cache The data storage receives the user's data request and detects the data hit rate of the user's data request hitting the cached data, that is, the probability that the data requested by the user is the cached data. The data hit rate can be calculated by obtaining the request hit rate of the cached data in the data request hitting the input data, and determining the average request rate corresponding to the data request. Then, the data hit rate of the cached data is calculated based on the request hit rate and the average request rate. The calculation formula of the data hit rate is as follows: The expected number of hits of object i is equal to i (i's average request rate) multiplied by the long-term probability of i in the cache to generate OHR ("request hit rate" object hit ratio). Then, the OHR predicted by the threshold parameter c is simply the ratio of expected hits to requests:

[0046] The calculation formula also includes the rate of data uploading and downloading in the cache queue, including the rate at which object i moves up to the top of the LRU list. The "uploading" rate r of the next request time interval is obtained by collecting aggregated statistics of object i in the previous stage. i The second parameter is the average rate at which object i is pushed down the LRU list. The "push down" rate c depends on the rate at which any object is moved to the top of the LRU list.

[0047] After calculating the data hit rate, the basic access threshold of the cache queue is iteratively adjusted, and the adjusted hit rate of the corresponding data request after the access threshold is adjusted is calculated. Then, the mapping relationship between the standard access threshold after adjusting the basic access threshold and the adjusted hit rate is determined, that is, the mapping relationship between different standard access thresholds and the corresponding adjusted hit rates.

[0048] Step S102 , repeating the iterative adjustment process, and adjusting the data attributes of the input data during the iterative adjustment process, combining the Markov model, and statistically determining the mapping relationship between the standard admission threshold and the adjusted hit rate under different data attributes.

[0049] Specifically, the iterative adjustment process in the above steps is repeated, and the data attributes of the input data are adjusted during the iterative process, such as the data domain corresponding to the data, the data size distribution, the number of data, etc., combined with the Markov model, and at the same time, under different data attributes, the mapping relationship between the standard access threshold after the basic access threshold is adjusted and the adjusted hit rate, that is, the mapping relationship between different standard access thresholds and the corresponding adjusted hit rates.

[0050] In addition, caching can be enabled / disabled for different data attributes, and a user-editable configuration file can be used to customize the cache data filtering priority. At the same time, a cache configuration method is provided, allowing users to choose the caching method based on the actual application scenario.

[0051] Step S103, after receiving the target data, obtain the data attributes of the target data, and based on the target mapping relationship corresponding to the data attributes of the target data, search for the target adjustment threshold corresponding to the maximum adjustment hit rate in the target mapping relationship, and filter the target data through the target adjustment threshold to determine the target cache data.

[0052] Specifically, after receiving the input target data, it is necessary to cache the input target data, that is, obtain the data attributes of the target data, and determine the target mapping relationship between the target data in the above steps based on the data attributes of the target data in combination with the Markov model. After determining the target mapping relationship, search according to the target mapping relationship set or target mapping relationship curve corresponding to the target mapping relationship, and search for the target adjustment threshold corresponding to the maximum adjustment hit rate, that is, the target adjustment threshold that best meets the user requirements. Based on the target adjustment threshold, perform preliminary data size screening on the target data to determine the target cache data after data size screening.

[0053] Step S104, obtain the data classification and data scenario corresponding to the target data, and determine the selection result and priority sorting result of the target data according to the data classification and data scenario, and select the target cache data based on the selection result, and prioritize the target cache data after data selection according to the priority sorting result, and determine the priority cache data after priority sorting.

[0054] Specifically, after determining the target cache data, determine the data classification corresponding to the target data and the data scenario of the data application, wherein the basis for data classification may include metadata, data size, data writing method and other basis for classification, and the specific classification situation may be, for example: A is adaptive adjustment of cache configuration, B is to classify metadata, C is to classify according to the written I / O size, D is the I / O writing method, E is priority configuration, F is default, and the default means according to the current cache policy, the corresponding application scenario may include the application scenario corresponding to the input data itself when it is applied, and then select the result to select the target cache data, wherein data selection is selective allocation of target data, such as metadata, data size, and data writing method. Two or more types of allocation can be selected for metadata, data size, and data writing method. The basis for selection may include the context of data scenarios, or manually selected metadata, data size, and data writing method type, and the target cache data after data selection is prioritized according to the priority sorting result. The priority sorting result can also refer to the context of data scenarios, or manually selected metadata, data size, and data writing method type to determine the priority cache data after priority sorting.

[0055] In addition, the corresponding historical data can be obtained based on the data attributes of the target data, and the historical data type and the corresponding historical data scenario corresponding to the data request frequency of the historical data can be detected. Then, the call sequence record of the data in the historical data can be obtained, and the game of the historical data can be determined according to the call sequence record. Then, the selection result of the target data can be determined based on the historical data type and the historical data scenario, and the call sequence record can be used to determine the priority sorting result.

[0056] Step S105, obtaining metadata in the priority cache data, counting the throughput corresponding to the metadata, filtering the metadata in the priority cache data based on the throughput, determining the final cache data after filtering, and placing the final cache data into a cache queue.

[0057] Specifically, the remaining metadata in the priority cache data is obtained. Metadata is data that describes object data. The throughput corresponding to the metadata is calculated. For the throughput calculation steps, Ceph is used as an example. Using object storage, XFS is used as the base file system, and 8+3 erasure coding (EC8+3) is used to store files. The file system of a single Ceph storage node typically contains hundreds of millions of small files. Retrieving one of these files requires collecting 8+3 erasure coded fragments and reassembling them into the original object. To retrieve each fragment, the file must first be tracked through the XFS inode to find the file on the disk hosting the corresponding XFS file system. Because there are hundreds of millions of files on disk, small (small-block) files must be tracked through 4-6 inode blocks before loading. This directly results in a latency six times longer than loading the small file itself and a throughput reduction of one-sixth of the overall potential. After calculating the throughput corresponding to the metadata, the metadata in the priority cache data is filtered based on the throughput. The final cached data is determined and queued into the cache queue according to the priority ranking results.

[0058] In addition, after determining the cache queue, when a target data request for the cache queue is received, when the number of requests for a single data in the target data request is less than a preset threshold, it means that it has no impact on the priority of the data in the current cache queue, and the data sorting of the final cached data in the cache queue will not be adjusted, so that the cache queue remains as it is, and the cache requirements of the target data can be met to the greatest extent.

[0059] An embodiment of the present invention provides a cache method based on adaptive adjustment of LRU differentiation, which configures a basic access threshold in a cache queue, and after filtering the input data through the basic access threshold to determine the cache data, calculates the data hit rate of the cache data when a data request is received, and iteratively adjusts the basic access threshold, calculates the adjusted hit rate, and determines a mapping relationship based on the adjusted standard access threshold and the adjusted hit rate; repeats the iterative adjustment process, and adjusts the data attributes of the input data during the iterative adjustment process, combines the Markov model, and determines a mapping relationship between the standard access threshold and the adjusted hit rate under different data attributes; when the target data is received, obtains the data attributes of the target data, and calculates the target hit rate based on the data attributes of the target data. The target mapping relationship is searched, and the target adjustment threshold corresponding to the maximum adjustment hit rate in the target mapping relationship is searched. The target data is filtered through the target adjustment threshold to determine the target cache data; the data classification and data scenario corresponding to the target data are obtained, and the selection result and priority sorting result of the target data are determined based on the data classification and data scenario, and the target cache data is selected based on the selection result, and the target cache data after data selection is prioritized according to the priority sorting result to determine the priority cache data after priority sorting; the metadata in the priority cache data is obtained, the throughput corresponding to the metadata is counted, and the metadata in the priority cache data is filtered based on the throughput to determine the final cache data after filtering, and the final cache data is queued into the cache queue. In this way, the optimal cache level for active data can be automatically determined, the object hit rate can be optimized, and the application execution speed can be improved. Through priority sorting and selective allocation, storage elements that can improve performance can be identified and cached. For metadata classification cache, the performance of specific scenarios can be improved, so that the cache queue can be flexibly and adaptively adjusted according to differentiated scenarios to meet user needs.

[0060] FIG2 is a cache system based on adaptive adjustment of LRU differentiation provided by an embodiment of the present invention, comprising: an iterative adjustment module S201, a mapping module S202, a screening module S203, a priority module S204, and a metadata module S205, wherein:

[0061] The iterative adjustment module S201 is used to configure a basic access threshold in the cache queue, and after filtering the input data through the basic access threshold to determine the cached data, calculate the data hit rate of the cached data when a data request is received, and iteratively adjust the basic access threshold, calculate the adjusted hit rate, and determine the mapping relationship based on the adjusted standard access threshold and the adjusted hit rate.

[0062] The mapping module S202 is used to repeat the iterative adjustment process, and adjust the data attributes of the input data during the iterative adjustment process, and determine the mapping relationship between the standard admission threshold and the adjustment hit rate under different data attributes by combining the Markov model.

[0063] The screening module S203 is used to obtain the data attributes of the target data after receiving the target data, and based on the target mapping relationship corresponding to the data attributes of the target data, search for the target adjustment threshold corresponding to the maximum adjustment hit rate in the target mapping relationship, and screen the target data through the target adjustment threshold to determine the target cache data.

[0064] The priority module S204 is used to obtain the data classification and data scenario corresponding to the target data, and determine the selection result and priority sorting result of the target data based on the data classification and data scenario, and select the target cache data based on the selection result, and prioritize the target cache data after data selection according to the priority sorting result to determine the priority cache data after priority sorting.

[0065] The metadata module S205 is used to obtain the metadata in the priority cache data, count the throughput corresponding to the metadata, filter the metadata in the priority cache data based on the throughput, determine the final cache data after filtering, and put the final cache data into the cache queue.

[0066] In one embodiment, the system further comprises:

[0067] The detection module is used to obtain corresponding historical data based on the data attributes of the target data, and detect the historical data type and the corresponding historical data scenario corresponding to the data request frequency of the historical data.

[0068] The acquisition module is used to obtain the call sequence record of the data in the historical data.

[0069] The determination module is used to determine the selection result of the target data based on the historical data type and the historical data scenario, and the calling sequence record determines the priority sorting result.

[0070] In one embodiment, the system further comprises:

[0071] The second acquisition module is used to obtain a request hit rate of the data request hitting the cached data in the input data, and determine an average request rate corresponding to the data request.

[0072] A calculation module is used to calculate the data hit rate of the cached data based on the request hit rate and the average request rate.

[0073] For the specific definition of the cache system with adaptive adjustment based on LRU differentiation, please refer to the definition of the cache method with adaptive adjustment based on LRU differentiation mentioned above, which will not be repeated here. Each module in the above-mentioned cache system with adaptive adjustment based on LRU differentiation can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0074] FIG3 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG3 , the electronic device may include: a processor 301, a memory 302, a communications interface 303, and a communication bus 304, wherein the processor 301, the memory 302, and the communications interface 303 communicate with each other through the communication bus 304. The processor 301 may call the logic instructions in the memory 302 to execute the following method: configuring a basic access threshold in the cache queue, and after filtering the input data through the basic access threshold to determine the cached data, calculating the data hit rate of the cached data when a data request is received, and iteratively adjusting the basic access threshold, calculating the adjusted hit rate after adjustment, and determining a mapping relationship based on the adjusted standard access threshold and the adjusted hit rate; repeating the iterative adjustment process, and adjusting the data attributes of the input data during the iterative adjustment process, combining the Markov model, and determining a mapping relationship between the standard access threshold and the adjusted hit rate under different data attributes; when receiving the target data, obtaining the data attributes of the target data, and determining a mapping relationship based on the target data attributes corresponding to the target data. The target mapping relationship is searched, and the target adjustment threshold corresponding to the maximum adjustment hit rate in the target mapping relationship is searched, and the target data is filtered by the target adjustment threshold to determine the target cache data; the data classification and data scenario corresponding to the target data are obtained, and the selection result and priority sorting result of the target data are determined according to the data classification and data scenario, and the target cache data is selected based on the selection result, and the target cache data after data selection is prioritized according to the priority sorting result, and the priority cache data after priority sorting is determined; the metadata in the priority cache data is obtained, the throughput corresponding to the metadata is counted, and the metadata in the priority cache data is filtered based on the throughput, and the final cache data after filtering is determined, and the final cache data is put into the cache queue.

[0075] In addition, the logic instructions in the above-mentioned memory 302 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0076] On the other hand, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the transmission methods provided by the above-mentioned embodiments, for example, including: configuring a basic access threshold in a cache queue, and after filtering the input data through the basic access threshold to determine the cached data, calculating the data hit rate of the cached data when a data request is received, and iteratively adjusting the basic access threshold, calculating the adjusted hit rate after adjustment, and determining a mapping relationship based on the adjusted standard access threshold and the adjusted hit rate; repeating the iterative adjustment process, and adjusting the data attributes of the input data during the iterative adjustment process, combining the Markov model, and statistically determining the mapping relationship between the standard access threshold and the adjusted hit rate under different data attributes; when the target data is received, obtaining the target data The method comprises the following steps: obtaining the data attributes of the target data, and based on the target mapping relationship corresponding to the data attributes of the target data, searching for the target adjustment threshold corresponding to the maximum adjustment hit rate in the target mapping relationship, filtering the target data through the target adjustment threshold to determine the target cache data; obtaining the data classification and data scenario corresponding to the target data, and determining the selection result and priority sorting result of the target data according to the data classification and data scenario, selecting the target cache data based on the selection result, and prioritizing the target cache data after data selection according to the priority sorting result, and determining the priority cache data after priority sorting; obtaining the metadata in the priority cache data, counting the throughput corresponding to the metadata, filtering the metadata in the priority cache data based on the throughput, determining the final cache data after filtering, and placing the final cache data into the cache queue.

[0077] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0078] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A cache method based on adaptive adjustment of LRU differentiation, characterized in that: include: A basic access threshold is configured in the cache queue, and after the input data is screened by the basic access threshold to determine the cache data, a data hit rate of the cache data is calculated when a data request is received, and the basic access threshold is iteratively adjusted to calculate the adjusted hit rate, and a mapping relationship is determined based on the adjusted standard access threshold and the adjusted hit rate; Repeating the iterative adjustment process, and adjusting the data attributes of the input data during the iterative adjustment process, combining the Markov model, and determining the mapping relationship between the standard admission threshold and the adjusted hit rate under different data attributes; After receiving the target data, obtaining the data attributes of the target data, and based on the target mapping relationship corresponding to the data attributes of the target data, searching for the target adjustment threshold corresponding to the maximum adjustment hit rate in the target mapping relationship, and screening the target data by the target adjustment threshold to determine the target cache data; Obtaining the data classification and data scenario corresponding to the target data, and determining the selection result and priority sorting result of the target data according to the data classification and data scenario, and performing data selection on the target cache data based on the selection result, and prioritizing the target cache data after the data selection according to the priority sorting result, and determining the priority cache data after the priority sorting; The metadata in the priority cache data is obtained, the throughput corresponding to the metadata is counted, the metadata in the priority cache data is screened based on the throughput, the final cache data after the screening is determined, and the final cache data is placed in a cache queue.

2. The cache method based on LRU differentiation and adaptive adjustment according to claim 1 is characterized in that: The determining of the selection result and the priority sorting result of the target data according to the data classification and the data scenario includes: Acquire corresponding historical data based on the data attributes of the target data, and detect the historical data type and the corresponding historical data scenario corresponding to the data request frequency of the historical data; Obtaining a call sequence record of data in the historical data; The selection result of the target data is determined based on the historical data type and the historical data scenario, and the calling sequence record determines the priority sorting result.

3. The cache method based on LRU differentiation and adaptive adjustment according to claim 1 is characterized in that: The calculating of the data hit rate of the cached data when the data request is received includes: Obtaining a request hit rate of the data request hitting the cached data in the input data, and determining an average request rate corresponding to the data request; A data hit rate of the cached data is calculated based on the request hit rate and the average request rate.

4. The cache method based on LRU differentiation and adaptive adjustment according to claim 1 is characterized in that: The counting of throughput corresponding to the metadata includes: The metadata is tracked through the index node, and the managed XFS file system finds the corresponding file of the index node on the disk, and determines the corresponding throughput based on the number of the index nodes.

5. The cache method based on LRU differentiation and adaptive adjustment according to claim 1 is characterized in that: After the final cache data is placed into the cache queue, the method further includes: When a target data request for the cache queue is received, when the number of requests for a single data in the target data request is less than a preset threshold, the data sorting of the final cached data in the cache queue is not adjusted.

6. The cache method based on LRU differentiation and adaptive adjustment according to claim 1, characterized in that: The calculation formula of the data hit rate includes: Among them, OHR(c) is the ratio of data hits to data requests, i is the object of the data request, r i is the migration rate of object i in the time interval between the data request in the previous stage and the next data request.

7. A cache system with adaptive adjustment based on LRU differentiation, characterized in that: The system comprises: An iterative adjustment module, configured to configure a basic admission threshold in a cache queue, and after filtering input data by the basic admission threshold to determine cache data, calculate a data hit rate of the cache data when a data request is received, and iteratively adjust the basic admission threshold, calculate an adjusted hit rate after adjustment, and determine a mapping relationship based on the adjusted standard admission threshold and the adjusted hit rate; A mapping module, used to repeat the iterative adjustment process, and adjust the data attributes of the input data during the iterative adjustment process, and determine the mapping relationship between the standard admission threshold and the adjustment hit rate under different data attributes by combining the Markov model; The filtering module is used to obtain the data attributes of the target data after receiving the target data, and Based on the target mapping relationship corresponding to the data attribute of the target data, searching for the target adjustment threshold corresponding to the maximum adjustment hit rate in the target mapping relationship, and screening the target data by the target adjustment threshold to determine the target cache data; A priority module, used to obtain the data classification and data scenario corresponding to the target data, and determine the selection result and priority sorting result of the target data according to the data classification and data scenario, and select the target cache data based on the selection result, and prioritize the target cache data after the data selection according to the priority sorting result, and determine the priority cache data after the priority sorting; The metadata module is used to obtain the metadata in the priority cache data, count the throughput corresponding to the metadata, filter the metadata in the priority cache data based on the throughput, determine the final cache data after filtering, and put the final cache data into a cache queue.

8. The cache system based on LRU differentiation and adaptive adjustment according to claim 7, characterized in that: The system further comprises: A detection module, configured to obtain corresponding historical data based on data attributes of the target data, and detect the historical data type and the corresponding historical data scenario corresponding to the data request frequency of the historical data; An acquisition module, used for acquiring a call sequence record of data in the historical data; A determination module is used to determine the selection result of the target data based on the historical data type and the historical data scenario, and the calling sequence record determines the priority sorting result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the cache method for adaptive adjustment based on LRU differentiation as claimed in any one of claims 1 to 6 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the cache method for adaptive adjustment based on LRU differentiation as claimed in any one of claims 1 to 6 are implemented.

Images