Data caching method, electronic device, storage medium and computer program product
By layering the data in memory, the first database with high data popularity caches high-hot data, and deletes the low-hot data to the second database, and caches compressed data in the third database, the problems of data reading delay and low efficiency in the prior art are solved, and more efficient data search and read and write performance are achieved.
Patent Information
- Application Number
- CN202411776659.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-05-06
AI Technical Summary
The prior art frequently eliminates data from memory and reads data from disk when reading data, resulting in increased data transmission delay and reduces service performance and data reading efficiency.
By layering the data in memory, the first database with high data popularity caches high-hot data, and deletes the low-hot data to the second database, and caches compressed data in the third database, and searches data from the high-hot database first.
The scope of single data search is narrowed, the hit rate and efficiency of data search is improved, the latency of I/O read and write, and the service performance is improved.
Smart Images
Figure CN119938705A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a data caching method, electronic equipment, storage medium and computer program product. Background Art
[0002] In the related art, when the data to be read is not cached in the memory, it is necessary to read and load the data from the disk into the memory, and then establish a data cache, which has poorer performance than reading directly from the memory. The commonly used way to improve data reading performance is to increase the memory capacity of the data server so that more data can be cached. However, the memory capacity of the server cannot be increased infinitely. When the amount of data reaches a certain level, it will be frequently eliminated from the memory and read from the disk, resulting in increased data transmission delays, reduced service performance and data reading efficiency. Summary of the invention
[0003] In view of this, embodiments of the present application provide a data caching method, an electronic device, a storage medium, and a computer program product.
[0004] The technical solution of the embodiment of the present application is implemented as follows:
[0005] An embodiment of the present application provides a data caching method, which includes: obtaining first data to be cached in a memory; the memory includes a first database and a second database, and the search priority of the first database is higher than that of the second database; if the amount of data cached in the first database has reached a first threshold, deleting the second data in the first database based on data heat, and caching the first data in the first database; caching the second data in the second database.
[0006] In the above solution, caching the second data in the second database includes: simplifying the second data to obtain simplified data corresponding to the second data; and caching the simplified data corresponding to the second data in the second database.
[0007] In the above scheme, the memory also includes a third database, and caching the second data in the second database includes: if the amount of data cached in the second database has reached a second threshold, compressing the data in the second database that supports compression; caching the compressed data in the third database, and deleting the data in the second database that supports compression; the search priority of data in the second database is higher than that of the third database; and caching the second data in the second database.
[0008] In the above scheme, after compressing all the data in the second database and caching them in the third database, the method further includes: if the data cached in the third database has reached a third threshold, deleting the third data in the third database based on data heat.
[0009] In the above scheme, the method includes: when receiving a data search request, searching for the data to be searched from the first database; if the data to be searched is not found in the first database, searching for the simplified data corresponding to the data to be searched from the second database; the simplified data is cached in the second database; if the simplified data corresponding to the data to be searched exists in the second database, restoring the data to be searched based on the simplified data, and deleting the simplified data corresponding to the data to be searched in the second database.
[0010] In the above scheme, the method also includes: if the simplified data corresponding to the data to be found does not exist in the second database, searching for compressed data of the simplified data corresponding to the data to be found from the third database in the memory; if compressed data of the simplified data corresponding to the data to be found exists in the third database, restoring the data to be found based on the compressed data, and deleting the compressed data in the third database.
[0011] In the above scheme, the deleting of the second data in the first database based on data heat includes: determining the time interval between all the data in the first database and the last use; deleting the data in the first database with the longest time interval between the last use, and the data with the longest time interval is the second data.
[0012] An embodiment of the present application also provides an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor is used to execute the steps of the method in the above scheme when running the computer program.
[0013] An embodiment of the present application further provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above scheme are implemented.
[0014] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of the method in the above solution when executed by a processor.
[0015] The embodiment of the present application obtains the first data to be cached in the memory, and the memory includes a first database and a second database. The search priority of the first database is higher than that of the second database. If the amount of data cached in the first database has reached the first threshold, the second data in the first database is deleted based on the data heat, and the first data is cached in the first database, and the second data is cached in the second database. The embodiment of the present application caches data in layers in the memory based on data heat, caches the first data with high data heat in the first database, and deletes the second data with low data heat in the first database to the second database. When reading and writing data, data is searched from the first database with a higher priority first, which reduces the scope of a single data search. Moreover, since the first database contains only high-heat data, the hit rate and efficiency of data search are improved, thereby reducing the delay of I / O reading and writing and improving service performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 A flowchart of a data caching method provided in an embodiment of the present application;
[0017] Figure 2 A flowchart of another data caching method provided in an embodiment of the present application;
[0018] Figure 3 A flowchart of a data search method provided in an embodiment of the present application;
[0019] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0020] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.
[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.
[0022] In the distributed file system in the related technology, when the required data information is not cached in the memory, it is necessary to read and load the data from the disk into the content, and then establish a data cache. Its performance is four orders of magnitude worse than reading directly from the memory. A common way to improve the performance of the file system is to cache more data.
[0023] In this regard, most related technologies adopt the following methods: adding more metadata servers (Meta Data Store, MDS) of the distributed file system (Ceph) to the cluster or increasing the server memory capacity to cache more data, but the above methods mainly have the following problems:
[0024] 1. The cost of adding new clusters or expanding the server capacity is too high. In addition, since the data in the distributed file system is large, it is unrealistic to increase its memory indefinitely. At the same time, when the amount of data reaches the order of one million or more, data will be frequently eliminated from the memory and read from the disk, which will cause the input / output (I / O) delay to increase significantly and fluctuate significantly, greatly reducing the service performance and having a bad impact on the user experience.
[0025] 2. When the MDS cluster is expanded, a large amount of data migration will occur, which will affect the normal business access performance of the storage system and reduce the availability of the storage system.
[0026] In view of the shortcomings of the above-mentioned related technologies, the embodiments of the present application provide a data caching method, which can cache data in layers based on heat to improve the efficiency of data search. In order to illustrate the technical solution described in the present application, a specific embodiment is used for description below.
[0027] Figure 1 A flow chart of a data caching method provided in an embodiment of the present application, which can be applied to a storage server, a distributed file system, or a storage memory of an electronic device, etc. Figure 1 , the method comprising:
[0028] Step 101, obtaining first data to be cached in a memory; the memory includes a first database and a second database, and the search priority of the first database is higher than that of the second database.
[0029] In some embodiments, the first data may be data that the user indicates needs to be stored, or data that the server requests to cache based on the popularity of the data in the current disk. Here, the first data may be metadata, a data packet, or a data set. Specifically, the popularity of the data may indicate the number of times the data is accessed within a period of time. The higher the frequency of access, the higher the popularity of the data. The server may periodically obtain the most popular data from the disk for caching, and when the user accesses the data, it may be accessed directly from the memory. The popularity of the data may also indicate the time when the data is accessed. The closer the access time is to the current time, the higher the data popularity.
[0030] In some embodiments, the memory can be divided into a first database and a second database. The sizes of the first database and the second database can be the same or different. Preferably, the size of the first database is larger than the second database. When cached data is obtained, it is cached in the first database first. Similarly, when the user searches for data, the first database is searched first. If the target data cannot be found in the first database, the target data is searched in the second database.
[0031] Step 102: If the amount of data cached in the first database has reached a first threshold, the second data in the first database is deleted based on data heat, and the first data is cached in the first database.
[0032] In some embodiments, the remaining capacity of the first database can be continuously monitored. If the amount of data cached in the first database is close to the maximum capacity, for example, it exceeds the first threshold, the second data in the first database can be deleted based on the data heat. Here, the first threshold can be set according to the actual situation, for example, it is set to 95% of the maximum capacity of the first database, etc. It should be noted that in some cases the system can pre-judge the capacity of the first database. For example, the current amount of data in the first database is 85% of the total capacity, but it may reach 98% after caching the first data. In this case, the data in the first database needs to be deleted in advance to reserve space for caching the first data.
[0033] In some embodiments, the second data in the first database can be deleted based on the least recently used (Least Recently Used, LRU) algorithm, for example, deleting data that has not been accessed for a long time. In some cases, the data in the first database can also be deleted in batches. After the data deletion is completed, the first data can be cached to the first data. If the first database has enough remaining capacity to cache the first data, the first data can also be directly cached to the first database. It is not required here to delete the data in the first database before caching the data. Data deletion can be performed only when the data in the first database reaches the first threshold or is about to reach the first threshold.
[0034] Step 103: cache the second data in the second database.
[0035] In some embodiments, before the second data is cached in the second database, the second data may be processed, for example, the second data may be streamlined, compressed, etc. to reduce the space occupied by the second data, and then cached in the second database.
[0036] The embodiment of the present application obtains the first data to be cached in the memory, and the memory includes a first database and a second database. The search priority of the first database is higher than that of the second database. If the amount of data cached in the first database has reached the first threshold, the second data in the first database is deleted based on the data heat, and the first data is cached in the first database, and the second data is cached in the second database. The embodiment of the present application caches data in layers in the memory based on data heat, caches the first data with high data heat in the first database, and deletes the second data with low data heat in the first database to the second database. When reading and writing data, data is searched from the first database with a higher priority first, which reduces the scope of a single data search. Moreover, since the first database contains only high-heat data, the hit rate and efficiency of data search are improved, thereby reducing the delay of I / O reading and writing and improving service performance.
[0037] In some embodiments, caching the second data in the second database includes:
[0038] Simplifying the second data to obtain simplified data corresponding to the second data;
[0039] The simplified data corresponding to the second data is cached in the second database.
[0040] In some embodiments, the second data may be simplified and then cached in the second database. Specifically, meaningless fields of the second data may be deleted. Of course, other data simplification methods may be used to simplify the second data. As long as the volume of the second data can be reduced and the original data can be restored based on the simplified data, the method can be applied to this embodiment.
[0041] Based on the above embodiment, the deleted second data is streamlined, so that more data can be cached in the memory, and because the heat of the data in the second database is lower than the heat of the data in the first database, the streamlined caching of low-heat data does not affect the reading of high-heat data.
[0042] In some embodiments, the memory further includes a third database, and caching the second data in the second database includes:
[0043] If the amount of data cached in the second database has reached a second threshold, compressing the data in the second database that supports compression;
[0044] Cache the compressed data in a third database, and delete the data in the second database that supports compression; the search priority of the data in the second database is higher than that of the third database;
[0045] The second data is cached in the second database.
[0046] In some embodiments, a third database may be further divided from the memory, and when the amount of data cached in the second database reaches a second threshold, the data in the second database that supports compression (compressible) needs to be compressed and cached in the third database, and the data in the second database that supports compression needs to be deleted. Here, the second threshold may be set based on the same logic as the first threshold.
[0047] In another embodiment, all data in the second database may be compressed, and data in the second database may be deleted based on data popularity in a similar manner to deleting data in the first database, and the deleted data may be compressed and cached in the third database. When searching for data, the search priority of the third database is the lowest, lower than that of the second database and the first database.
[0048] In some embodiments, a fourth database, a fifth database or even multiple databases may be set up for caching data. It is understandable that the data heat of the data cached in these databases decreases successively, and the compression degree increases successively. The specific number of databases to be set can be determined according to actual conditions.
[0049] Based on the above embodiment, when the amount of data in the second database reaches the second threshold, the data in the second database is compressed, cached in the third database, and the data in the second database is deleted. The operation of deleting the data in the second database will not be triggered until the amount of data cached in the second database reaches the second threshold again, thereby reducing system processing requirements and caching more data using the same memory space. The third database has lower popularity and the lowest search priority, and the cached data is less in demand, which will not affect the search for popular data.
[0050] In some embodiments, after compressing all the data in the second database and caching them in the third database, the method further includes:
[0051] If the data cached in the third database has reached a third threshold, the third data in the third database is deleted based on the data heat.
[0052] In some embodiments, when the amount of data cached in the third database reaches a third threshold, the data in the third database needs to be deleted. Specifically, the third threshold can be consistent with the setting logic of the first threshold and the second threshold in the aforementioned embodiments, and the data in the third database can also be deleted based on the LRU algorithm, and the third data with the lowest heat is deleted. Here, the third data can be a single data or a data packet. In some cases, the data in the third database can also be cleared regularly.
[0053] In some embodiments, if the capacities of the first database, the second database, and the third database all reach corresponding thresholds, the data in the third database is deleted first, followed by the second database, and finally the data in the first database, and the caching and deletion operations of the above multiple databases can be performed independently.
[0054] Based on the above embodiment, when the amount of data in the third database reaches the third threshold, the data with the lowest popularity in the third database can be deleted to free up memory space, and the caching and deletion operations between databases can be performed independently without affecting each other, thereby avoiding the impact of a large amount of data migration at the same time on server performance.
[0055] In some embodiments, the method comprises:
[0056] When receiving a data search request, searching the first database for the data to be searched;
[0057] If the to-be-searched data is not found in the first database, searching the second database for simplified data corresponding to the to-be-searched data; the second database has simplified data cached;
[0058] If the simplified data corresponding to the data to be found exists in the second database, the data to be found is restored based on the simplified data, and the simplified data corresponding to the data to be found in the second database is deleted.
[0059] In some embodiments, when it is necessary to search for data from the memory, the data to be searched can be first searched from the first database in the memory. The search method here is not limited to single table search, index search, random search, etc. If the data to be searched cannot be found in the first database, it will turn to the second database to search for the data to be searched. Since the data cached in the second database is streamlined data, it is necessary to search the streamlined data corresponding to the data to be searched in the second database.
[0060] If simplified data corresponding to the data to be searched is found in the second database, the simplified data is extracted and the data to be searched is restored based on the simplified data. The specific restoration process may be the reverse process of the simplification in the aforementioned embodiment, which will not be described here. The simplified data corresponding to the search data in the second database is deleted.
[0061] In some embodiments, if the simplified data is searched multiple times within a period of time, it means that the popularity of the to-be-searched data corresponding to the simplified data increases, and the to-be-searched data corresponding to the simplified data can be cached in the first database.
[0062] Based on the above embodiment, when searching for data to be searched, it is preferred to start from the first database that caches data with higher popularity, so that the search range is smaller in a single search, and the search efficiency is higher.
[0063] In some embodiments, the method further comprises:
[0064] If the simplified data corresponding to the data to be found does not exist in the second database, searching the third database in the memory for compressed data of the simplified data corresponding to the data to be found;
[0065] If compressed data of the simplified data corresponding to the data to be found exists in the third database, the data to be found is restored based on the compressed data, and the compressed data in the third database is deleted.
[0066] In some embodiments, if the reduced data corresponding to the data to be found does not exist in the second database, it is necessary to search for the compressed data corresponding to the reduced data in the third database in the memory, and the specific search method can be the same as the search method in the above embodiment. If the compressed data with the reduced data corresponding to the search data is found in the third database, the compressed data is extracted and restored based on the compressed data to obtain the data to be found. At the same time, the compressed file can be deleted from the third database. In some cases, the restored data to be found can also be cached in the first database.
[0067] Based on the above embodiment, the data to be searched is searched from multiple databases in sequence, and the amount of data involved in a single search operation is smaller, which can improve the efficiency of data search. After the low-temperature data is found, it is restored and the cache in the memory is deleted to release memory space to store other data.
[0068] In some embodiments, deleting the second data in the first database based on data heat includes:
[0069] Determine the time interval from the last use of all data in the first database;
[0070] The data in the first database with the longest time interval from being last used is deleted, and the data with the longest time interval is the second data.
[0071] In some embodiments, the popularity of data can be determined based on the time interval between the use of data in the database, that is, the more frequently the data is used and the smaller the use interval, the higher the popularity of the data. When deleting data, the lowest popularity data in the database can be deleted, or a popularity threshold can be set. For example, if the data has not been used for more than a week, all data with a popularity lower than the threshold can be deleted to achieve batch deletion of data in the database. For the first database, the time interval between the last use of all data in the first database can be deleted, and the second data with the largest time interval can be deleted.
[0072] Based on the above embodiment, the data popularity is determined by the frequency with which the data is used, and the data to be deleted is determined based on this. The data with higher popularity is always stored in the first database, which can hit the data to be searched with a higher probability, thereby improving the efficiency of data search.
[0073] Figure 2 A flow chart of another data caching method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the method includes:
[0074] Step 201: Whether the amount of L0 layer data exceeds a threshold.
[0075] Here, the L0 layer is the storage layer in the memory, corresponding to the first database in the aforementioned embodiment. When caching data in the memory, it is preferentially cached to the L0 layer. At the same time, it is also necessary to determine whether the amount of data in the L0 layer exceeds the threshold. If it does not exceed, it is continuously monitored. If it exceeds, it enters step 202.
[0076] Step 202: The LRU traverses the L0 layer and eliminates data that is not occupied by services.
[0077] The data in the L0 layer is traversed based on LRU, and the data that has not been used for a long time is eliminated. The data that is being used does not need to be traversed here.
[0078] Step 203: Eliminated data is streamlined and cached in the L1 layer.
[0079] The data eliminated in step 202 is streamlined and cached to the L1 layer, where the L1 layer is also a storage layer in the memory. Corresponding to the second database in the above embodiment, the search priority of the L1 layer is lower than that of the L0 layer.
[0080] Step 204, delete the data eliminated by the L0 layer.
[0081] After completing step 203, the eliminated data in the L0 layer is deleted to release the space in the L0 layer.
[0082] Step 205: Whether the L1 layer data volume exceeds the threshold.
[0083] The threshold value set at the L1 layer here can be based on the same setting logic as that of the L0 layer, for example, set to 90% of the maximum capacity. If the threshold value is not exceeded, the data volume is continuously monitored. If the threshold value is exceeded, the process proceeds to step 206.
[0084] Step 206, traverse all data in the L1 layer.
[0085] Here, statistics can be collected on all data of the L1 layer to prepare for step 207 .
[0086] Step 207: compress the L1 layer compressible data by using a compression module.
[0087] Compress the compressible data of the L1 layer.
[0088] Step 208: Delete the compressible data of the L1 layer.
[0089] Step 209: Cache the compressed data to the L2 layer.
[0090] The compressed data obtained in step 207 is cached to the L2 layer, where the L2 layer is a storage layer of the memory, which may correspond to the third database in the aforementioned embodiment.
[0091] Based on the above embodiment, the memory is layered, data is eliminated based on LRU, hot data is kept in the memory, and the hit rate of data search is improved.
[0092] Figure 3 A flowchart of a data search method provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the method includes:
[0093] Step 301, obtaining cache data information.
[0094] The cached data information here may refer to the relevant information of the data to be found.
[0095] Step 302, search in L0 layer.
[0096] The data to be searched is searched preferentially in the L0 layer of the memory. If the data is found, the process goes to step 308 . If not found, the process goes to step 303 .
[0097] Step 303, search in L1 layer.
[0098] When the to-be-searched data is not found in the L0 layer, the search is performed in the L1 layer, which has the reduced data cached therein. If the reduced data corresponding to the to-be-searched data is found, the process proceeds to step 305 , and if not, the process proceeds to step 304 .
[0099] Step 304, search at the L2 layer.
[0100] If the data to be searched is not found in the L1 layer, the search continues in the L2 layer. The L2 layer caches compressed data. If the compressed data of the reduced data corresponding to the data to be searched is found, the process proceeds to step 306. If not found, the process proceeds to step 307.
[0101] Step 305: extract data and delete corresponding simplified data.
[0102] The simplified data corresponding to the data to be searched is extracted, and the simplified data in the L1 layer is deleted.
[0103] Step 306, decompress the data and delete the corresponding compressed data.
[0104] The compressed data of the simplified data corresponding to the data to be found is decompressed to obtain the data to be found, and the compressed data corresponding to the L2 layer is deleted.
[0105] Step 307, search from other mds or load from disk.
[0106] When the data to be found cannot be found in the current server or memory, it can be searched from other servers, memory or MDS, or directly loaded from the disk. When searching for the data to be found from other servers, memory or MDS, the same search method as above can be used.
[0107] Step 308, obtaining cache data.
[0108] Get the found data to be searched.
[0109] Step 309: Process related service requests.
[0110] Based on the acquired data to be searched, process related business requests, such as responding to user requests.
[0111] Based on the above embodiment, when searching for data, the search is performed through a jump table and asynchronous compression is performed, which does not affect the performance of the main process.
[0112] The embodiment of the present application also provides a data caching device, which corresponds to the above-mentioned data caching method, and each step in the above-mentioned data caching method embodiment is also completely applicable to the embodiment of the present device.
[0113] The data caching device comprises:
[0114] The acquisition module is used to acquire first data to be cached in a memory; the memory includes a first database and a second database, and the search priority of the first database is higher than that of the second database.
[0115] The processing module is used to eliminate the second data in the first database based on data heat and cache the first data in the first database when the amount of data cached in the first database has reached a first threshold.
[0116] The processing module is further configured to cache the second data in the second database.
[0117] The processing module is further used to perform simplification processing on the second data to obtain simplification data corresponding to the second data; and cache the simplification data corresponding to the second data into the second database.
[0118] The processing module is also used to compress the data that supports compression in the second database if the amount of data cached in the second database has reached a second threshold; cache the compressed data in a third database, and delete the data that supports compression in the second database; the search priority of the data in the second database is higher than that of the third database; and cache the second data in the second database.
[0119] The processing module is further configured to eliminate the third data in the third database based on data heat when the data cached in the third database has reached a third threshold.
[0120] The device further comprises a search module, which is used to search for the data to be searched from the first database when receiving a data search request;
[0121] If the to-be-searched data is not found in the first database, searching the second database for simplified data corresponding to the to-be-searched data; the second database has simplified data cached;
[0122] If the simplified data corresponding to the data to be found exists in the second database, the data to be found is restored based on the simplified data, and the simplified data corresponding to the data to be found in the second database is deleted.
[0123] The search module is also used to search for compressed data of the simplified data corresponding to the data to be searched from the third database in the memory when there is no simplified data corresponding to the data to be searched in the second database; if there is compressed data of the simplified data corresponding to the data to be searched in the third database, restore the data to be searched based on the compressed data, and delete the compressed data in the third database.
[0124] The processing module is further used to determine the time interval between the last use of all data in the first database and the last use of all data in the first database;
[0125] Eliminate the data in the first database that has the longest time interval from being last used, and the data with the longest time interval is the second data.
[0126] In actual application, the acquisition module, processing module, and search module can be implemented by a processor in the electronic device. Of course, the processor needs to run the computer program in the memory to implement its functions.
[0127] It should be noted that: when the device provided in the above embodiment is applied, only the division of the above program modules is used as an example. In actual application, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the device and method embodiments provided in the above embodiment belong to the same concept and will not be repeated here.
[0128] Based on the hardware implementation of the above program modules and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an electronic device. Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the electronic device includes:
[0129] Communication interface 401, capable of exchanging information with other devices such as network devices;
[0130] The processor 402 is connected to the communication interface 401 to implement information exchange with other devices and is used to execute the method provided by one or more technical solutions when running a computer program. The computer program is stored in the memory 403.
[0131] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 404. It can be understood that the bus system 404 is used to realize the connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 4 Various buses are labeled as bus system 404 .
[0132] The memory 403 in the embodiment of the present application is used to store various types of data to support the operation of the computer device. Examples of such data include: any computer program used to operate on the electronic device.
[0133] It can be understood that the memory 403 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), direct memory bus random access memory (DRRAM). The memory described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memory.
[0134] The method disclosed in the above embodiment of the present application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the program in the memory and completes the steps of the above method in combination with its hardware.
[0135] Optionally, when the processor 402 executes the program, it implements the corresponding processes implemented by the computer device in each method of the embodiments of the present application, which will not be described here for the sake of brevity.
[0136] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a first memory storing a computer program, and the computer program can be executed by a processor of a computer device to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.
[0137] In the several embodiments provided in the present application, it should be understood that the disclosed devices, computer equipment and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0138] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
[0139] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0140] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.
[0141] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.
[0142] In an exemplary embodiment, the embodiment of the present application further provides a computer program product, including a computer program, which can be executed by the processor 402 of the electronic device to complete the steps described in the data caching method in the embodiment of the present application.
[0143] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0144] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.
[0145] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A data caching method, characterized in that: The method comprises: Acquire first data to be cached in a memory; the memory includes a first database and a second database, and the search priority of the first database is higher than that of the second database; If the amount of data cached in the first database has reached a first threshold, deleting the second data in the first database based on data popularity, and caching the first data in the first database; The second data is cached in the second database.
2. The method according to claim 1, characterized in that: The step of caching the second data in the second database includes: Simplifying the second data to obtain simplified data corresponding to the second data; The simplified data corresponding to the second data is cached in the second database.
3. The method according to claim 1, characterized in that: The memory further includes a third database, and caching the second data in the second database includes: If the amount of data cached in the second database has reached a second threshold, compressing the data in the second database that supports compression; Cache the compressed data in a third database, and delete the data in the second database that supports compression; the search priority of the data in the second database is higher than that of the third database; The second data is cached in the second database.
4. The method according to claim 3, characterized in that: After compressing all the data in the second database and caching them in the third database, the method further includes: If the data cached in the third database has reached a third threshold, the third data in the third database is deleted based on the data heat.
5. The method according to claim 1 or 2, characterized in that: The method comprises: When receiving a data search request, searching the first database for the data to be searched; If the to-be-searched data is not found in the first database, searching the second database for simplified data corresponding to the to-be-searched data; the second database has simplified data cached; If the simplified data corresponding to the data to be found exists in the second database, the data to be found is restored based on the simplified data, and the simplified data corresponding to the data to be found in the second database is deleted.
6. The method according to claim 5, characterized in that: The method further comprises: If the simplified data corresponding to the data to be found does not exist in the second database, searching the third database in the memory for compressed data of the simplified data corresponding to the data to be found; If compressed data of the simplified data corresponding to the data to be found exists in the third database, the data to be found is restored based on the compressed data, and the compressed data in the third database is deleted.
7. The method according to claim 1, characterized in that: The deleting the second data in the first database based on the data heat includes: Determine the time interval from the last use of all data in the first database; The data in the first database with the longest time interval from being last used is deleted, and the data with the longest time interval is the second data.
8. An electronic device, characterized in that: include: A processor and a memory for storing a computer program that can be executed on the processor, wherein: The processor is used to execute the steps of the method according to any one of claims 1 to 7 when running a computer program.
9. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.