Data query method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a data query method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: receiving a data access request, wherein the data access request comprises a data identifier of target data; in response to the data access request, according to the data identifier, querying whether the target data exists in a plurality of hierarchical cache spaces, different hierarchical cache spaces corresponding to different data access speeds; and if the target data does not exist in the plurality of hierarchical cache spaces, querying the target data from a disk. By adopting the method, the storage performance of the cache architecture can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, and in particular to a data query method and device, computer equipment, a storage medium and a program product. BACKGROUND

[0002] With the increasing requirements of data processing capabilities, the storage architecture of data storage space and related hardware facilities are constantly innovating and evolving, and the existing data storage scheme gradually fails to adapt to the storage requirements of various application scenarios. As an important part of the storage system, by storing data in the cache, the data scheduling strategy can be optimized, the I / O processing efficiency can be significantly improved, the system resource consumption can be reduced, and the storage performance can be comprehensively improved.

[0003] However, the traditional cache architecture design has a storage performance bottleneck. SUMMARY

[0004] Therefore, it is necessary to provide a data query method, device, computer equipment, storage medium and program product capable of improving the storage performance of the cache architecture to solve the above technical problems.

[0005] In a first aspect, the present application provides a data query method. The method comprises:

[0006] receiving a data access request, wherein the data access request comprises a data identifier of target data;

[0007] in response to the data access request, querying whether the target data exists in a plurality of hierarchical cache spaces according to the data identifier, wherein different hierarchical cache spaces correspond to different data access speeds;

[0008] if the target data does not exist in the plurality of hierarchical cache spaces, querying the target data from a disk.

[0009] In one embodiment, the plurality of hierarchical cache spaces comprises a dynamic cache space and a front-end cache space, and the querying whether the target data exists in the plurality of hierarchical cache spaces according to the data identifier comprises:

[0010] querying whether the target data exists in the dynamic cache space according to the data identifier, wherein the dynamic cache space is used for storing data according to data hotness;

[0011] if the target data exists in the dynamic cache space, obtaining the target data from the dynamic cache space;

[0012] If the target data does not exist in the dynamic cache space, it is determined whether the target data exists in the front-end cache space according to the data identifier, the front-end cache space being configured to store front-end data.

[0013] In one of the embodiments, before the target data is queried from the disk, the method further comprises:

[0014] According to the data identifier, a physical address of the target data in the disk is obtained from a preset metadata table.

[0015] According to the physical address, it is determined whether the target data exists in a target cache space, the target cache space being configured to store deduplicated data and persistent data in the disk.

[0016] In one of the embodiments, the determining whether the target data exists in the target cache space according to the physical address comprises:

[0017] According to the physical address, it is determined whether a target physical address identical to the physical address exists in a fingerprint index table in a deduplicated cache space included in the target cache space.

[0018] If the target physical address exists in the fingerprint index table, the target data is obtained from the deduplicated cache space.

[0019] If the target physical address does not exist in the fingerprint index table, it is determined whether the target physical address exists in a persistent cache space included in the target cache space according to the physical address.

[0020] In one of the embodiments, the method further comprises:

[0021] If the target physical address exists in the persistent cache space, the target data is obtained from the persistent cache space.

[0022] The querying the target data from the disk comprises:

[0023] If the target physical address does not exist in the persistent cache space, the target data is queried from the disk according to the physical address.

[0024] In one of the embodiments, after the target data is queried and obtained from the disk, the method further comprises:

[0025] The remaining storage capacity of the dynamic cache space is queried.

[0026] If the remaining storage capacity is greater than or equal to a preset storage capacity threshold, the target data is written into the dynamic cache space.

[0027] If the remaining storage capacity is less than the storage capacity threshold, performing a garbage collection operation on the dynamic cache space, and writing the target data into the dynamic cache space after performing the garbage collection operation.

[0028] In a second aspect, the present application provides a data query device. The device comprises:

[0029] a receiving module configured to receive a data access request, the data access request comprising a data identifier of target data;

[0030] a first query module configured to, in response to the data access request, query whether the target data exists in a plurality of hierarchical cache spaces according to the data identifier, different hierarchical cache spaces corresponding to different data access speeds;

[0031] a second query module configured to, if the target data does not exist in the plurality of hierarchical cache spaces, query the target data from a disk.

[0032] In a third aspect, the present application provides a computer device, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method of the first aspect when executing the computer program.

[0033] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, and the computer program implementing the steps of the method of the first aspect when executed by a processor.

[0034] In a fifth aspect, the present application provides a computer program product, the computer program product comprising a computer program, and the computer program implementing the steps of the method of the first aspect when executed by a processor.

[0035] The data query method, device, computer device, storage medium and program product, the server first receives a data access request including a data identifier of target data, then, in response to the data access request, queries whether the target data exists in the plurality of hierarchical cache spaces according to the data identifier, and then, if the target data does not exist in the plurality of hierarchical cache spaces, queries the target data from the disk. Since the data access speeds corresponding to different hierarchical cache spaces are different, hierarchical management of data is implemented. In this way, the server can improve the efficiency of data identifier hits and reduce data access time by preferentially accessing the plurality of hierarchical cache spaces. In the case where the target data does not exist in the plurality of hierarchical cache spaces, the target data is queried from the disk, thereby improving the overall query efficiency of the target data, reducing the number of direct accesses to the disk and I / O overhead, enabling the server to respond to more data access requests, improving throughput, and further improving the storage performance of the cache architecture. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0037] Figure 1 An application environment diagram of the data query method in an embodiment;

[0038] Figure 2 A flowchart of the data query method in an embodiment;

[0039] Figure 3 A flowchart of step 203 in an embodiment;

[0040] Figure 4 A flowchart of the data query method in another embodiment;

[0041] Figure 5 A flowchart of step 402 in an embodiment;

[0042] Figure 6 A flowchart of the data query method in another embodiment;

[0043] Figure 7 A flowchart of the data query method in another embodiment;

[0044] Figure 8 A block diagram of the data query device in an embodiment;

[0045] Figure 9Figure 1 is a diagram of an internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] For the purposes of the present application, the technical solutions and advantages thereof are more clearly apparent, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application.

[0047] As an important component in the storage system, the cache can optimize the data scheduling strategy, significantly improve the I / O processing efficiency, reduce the system resource consumption, and thus comprehensively improve the comprehensive performance indicators of the storage solution, through deep combination with specific business logic. The traditional cache architecture design, whether based on hash table, radix tree or other classic data structures, has its own performance bottleneck and applicable scenario limitation when used as the core component of the storage system cache.

[0048] In the performance optimization process of the storage system, the data structure design and selection of the cache system mainly follow the following core principles based on the actual business scenario requirements: ①Optimize memory access efficiency: on the premise of ensuring the integrity of business functions, preferentially use data structures with shorter memory addressing paths, reduce the number of pointer jumps in the data access process, and thus significantly improve the access efficiency of cache data. ②Implement intelligent caching of hot data: in the face of massive data scenarios, establish an efficient hot spot recognition mechanism to ensure that system resources preferentially cache data with high access frequency, and establish a reasonable elimination strategy to release resources occupied by low-frequency access data in a timely manner. ③Reduce the number of read disks: the performance of disk access is 100 times lower than that of memory access. In order to improve the data access performance of the storage machine, the number of disk accesses is minimized, the overall path of finding data is shortened, and the access performance of the storage machine is improved.

[0049] The existing data cache architecture usually adopts data storage structures such as LSM tree (Log-Structured Merge Tree), radix tree, hash table, etc. However, the read performance of the LSM tree is unstable, and the query may need to check the MemTable and multiple SSTable files layer by layer; the radix tree is difficult to fully utilize the cache prefetch function of modern processors; the memory access pointer of the data structure of the hash table needs to jump, increasing the data access delay.

[0050] In view of this, the application provides a data query method. A server first receives a data access request including a data identifier of target data, then, in response to the data access request, queries whether the target data exists in a plurality of hierarchical cache spaces according to the data identifier, and then, if the target data does not exist in the plurality of hierarchical cache spaces, queries the target data from a disk. Since the data access speeds of different hierarchical cache spaces are different, hierarchical management of data is achieved. In this way, the server can improve the efficiency of data identifier hits and reduce data access time by preferentially accessing the plurality of hierarchical cache spaces. In the case where the target data does not exist in the plurality of hierarchical cache spaces, the target data is queried from the disk, thereby improving the overall query efficiency of the target data, reducing the number of direct accesses to the disk and I / O overhead, enabling the server to respond to more data access requests, improving throughput, and further improving the storage performance of the cache architecture.

[0051] The data query method provided by the application embodiment can be applied to an implementation environment as shown in Figure 1 The implementation environment includes a server, which can be implemented by an independent server or a server cluster composed of a plurality of servers, and a data storage system that can store data required to be processed by the server. The data storage system can be integrated on the server, or placed on a cloud or other network server.

[0052] In other possible implementation manners, the data query method provided by the application embodiment can also be applied to a terminal. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc.

[0053] In an exemplary embodiment, as shown in Figure 2 A data query method is provided. The method is applied to a server in Figure 1 for example, and includes the following steps 201-203:

[0054] Step 201, receiving a data access request.

[0055] The data access request includes a data identifier of target data.

[0056] Optionally, the data access request can be an instruction for accessing data sent by a user to the server through a terminal. The target data refers to data that the user needs to access, for example, the user can query, modify, add, or delete the target data through the data access request. The data identifier is an identifier for uniquely representing data.

[0057] In the embodiment, the server can receive the data access request sent by the terminal through the network, and obtain the data identifier of the target data in the data access request by analyzing the data access request.

[0058] In step 202, in response to the data access request, whether the target data exists in the plurality of hierarchical cache spaces is queried according to the data identifier.

[0059] In the embodiment, the plurality of hierarchical cache spaces correspond to different data access speeds. For example, the hierarchical cache spaces can include a high-speed cache space and a low-speed cache space. The high-speed cache space can store frequently accessed data, and the low-speed cache space can store infrequently accessed data.

[0060] It should be noted that, due to limited memory resources, a differentiated data management mechanism can be designed in the storage system to divide the cache space into a plurality of different cache spaces. In the embodiment, the data can be stored in different hierarchical cache spaces according to the data access speed. For example, data with high access frequency can be allocated to a hierarchical cache space with high data access speed, and data with low access frequency can be allocated to a hierarchical cache space with low data access speed.

[0061] In this way, when accessing data, the plurality of hierarchical cache spaces can be accessed respectively to improve the data access speed. For example, the access order can be determined according to the order of the access speeds of the hierarchical cache spaces, and the corresponding hierarchical cache spaces can be accessed in turn according to the access order to improve the efficiency of the entire query process.

[0062] In the embodiment, the server can respond to the data access request, and in the response process, whether the target data exists in each hierarchical cache space can be determined according to the data identifier.

[0063] Optionally, if the data identifier exists in each hierarchical cache space, it can be determined that the target data exists in each hierarchical cache space, and if the data identifier does not exist in each hierarchical cache space, it can be determined that the target data does not exist in each hierarchical cache space.

[0064] In step 203, if the target data does not exist in the plurality of hierarchical cache spaces, the target data is queried from the disk.

[0065] It can be understood that if the target data does not exist in the plurality of hierarchical cache spaces, it can be determined that the data exists on the disk, and therefore the target data can be queried from the disk.

[0066] In the embodiment, the server can determine that the target data does not exist in the plurality of hierarchical cache spaces in the case that the data identifier does not exist in the plurality of hierarchical cache spaces, and then query the target data from the disk according to the data identifier.

[0067] As a possible implementation, the server can further acquire the target data from the plurality of hierarchical cache spaces and return to the terminal in the case that the target data exists in the plurality of hierarchical cache spaces, or the server can further acquire the target data from the disk and return to the terminal in the case that the target data does not exist in the plurality of hierarchical cache spaces.

[0068] The data query method described above, the server first receives a data access request including a data identifier of target data, and then in response to the data access request, queries whether the target data exists in the plurality of hierarchical cache spaces according to the data identifier, and then if the target data does not exist in the plurality of hierarchical cache spaces, queries the target data from the disk. Since the data access speeds corresponding to different hierarchical cache spaces are different, hierarchical management of data is achieved. In this way, the server can improve the efficiency of data identifier hits by preferentially accessing the plurality of hierarchical cache spaces, reduce data access time, and in the case that the target data does not exist in the plurality of hierarchical cache spaces, query the target data from the disk again, thereby improving the overall query efficiency of the target data, reducing the number of direct accesses to the disk and I / O overhead, enabling the server to respond to more data access requests, improving throughput, and further improving the storage performance of the cache architecture.

[0069] In one embodiment, the plurality of hierarchical cache spaces includes a dynamic cache space and a front-end cache space, and the data query method is based on Figure 2 As shown in the embodiment, refer to Figure 3 The embodiment relates to a process in which the server queries whether target data exists in the plurality of hierarchical cache spaces according to a data identifier. As shown in Figure 3 Step 203 includes Figure 3 Steps 301-303 shown in the embodiment.

[0070] Step 301: Query whether the target data exists in the dynamic cache space according to the data identifier.

[0071] The dynamic cache space is used for storing data according to data heat. The dynamic cache space has a higher storage cost and limited capacity, and is suitable for storing hotspot data. For example, data with a higher access frequency can be determined as hot data, and data with a lower access frequency can be determined as cold data. The hot data is stored in the dynamic cache space, and the data in the dynamic cache space is dynamically adjusted according to the access frequency.

[0072] As a possible implementation, the data can be stored in the form of KV key-value pairs in a hash table in the dynamic cache space. The data identifier can be mapped to the K value in the KV key-value pair.

[0073] In this embodiment, the server can first standardize the data identifier in a standard format to obtain a standardized data identifier, then call the interface of the dynamic cache space to query the data identifier consistent with the standardized data identifier in the dynamic cache space, and then determine whether the target data exists in the dynamic cache space according to the query result.

[0074] Step 302, if the target data exists in the dynamic cache space, the target data is obtained from the dynamic cache space.

[0075] In this embodiment, the server can determine that the target data exists in the dynamic cache space when the data identifier consistent with the standardized data identifier exists in the dynamic cache space, and then obtain the target data through the interface of the dynamic cache space.

[0076] Step 303, if the target data does not exist in the dynamic cache space, whether the target data exists in the front-end cache space is queried according to the data identifier.

[0077] The front-end cache space is used to store front-end data. The front-end cache space (such as browser cache, client local storage) has relatively large capacity and low storage cost, but is limited by network transmission or client performance, and the access speed of the front-end cache space is lower than that of the dynamic cache space (millisecond level). However, the front-end cache space still has a high hit rate. Therefore, when the dynamic cache space is not hit, the front-end cache space can be queried to determine whether the target data exists in the front-end cache space with lower delay, so as to avoid the data access request from concentrating on the server and dispersing the system load of the server.

[0078] As a possible implementation, the data can be stored in the form of memtable and Immutable based on LSM tree in the front-end cache space.

[0079] In this embodiment, the server can determine that the target data does not exist in the dynamic cache space when the data identifier consistent with the standardized data identifier does not exist in the dynamic cache space, and then query whether the target data exists in the front-end cache space according to the data identifier.

[0080] In this embodiment, the server first queries whether the target data exists in the dynamic cache space according to the data identifier, then acquires the target data from the dynamic cache space in the case that the target data exists in the dynamic cache space, and queries whether the target data exists in the front-end cache space according to the data identifier in the case that the target data does not exist in the dynamic cache space. Since the dynamic cache space stores data according to data popularity and the front-end cache space is used to store front-end data, the server first queries the target data from the dynamic cache space, which can improve the hit efficiency of the data identifier, and in the case of dynamic cache miss, the target data can be returned with low latency by combining the front-end cache, so that the disk resource consumption is reduced and the load of the server is lowered. In this way, the dynamic cache space and the front-end cache space can form a complementary relationship, the best balance between access speed and hit rate is achieved in the case of limited server resources, the overall performance of the front-end business cache is significantly improved, the utilization rate of the cache resources is optimized while the timeliness of data is ensured.

[0081] In one embodiment, based on the above embodiment, referring to Figure 4 The embodiment relates to a process of how to query whether target data exists in a target cache space before the server queries the target data from a disk, as shown in Figure 4 The server can perform steps 401 and 402 to implement the process.

[0082] Step 401: According to the data identifier, the physical address of the target data in the disk is acquired from a preset metadata table.

[0083] It should be noted that when the data of the front-end business is flushed to the disk, corresponding physical description information of the data is generated. The description information of the data on the disk is called metadata. The metadata is stored in a storage engine and can be stored in different metadata tables according to types. When responding to a data access request, if a multi-level cache space does not hit, the metadata table can be queried. In order to speed up the reading of the metadata and further improve the overall data access efficiency, a global cache mechanism is provided for the metadata storage to ensure that the metadata resides in the memory. The cache is implemented based on a PageCache (page cache), and the memory of the cache is mainly sourced from a cache pool. The cache pool dynamically allocates memory for the metadata table according to the priority of different storage tables.

[0084] The physical address in the disk is used to uniquely identify the position of a specific data block on a disk storage medium, and is a core address form for interaction between an operating system, a file system and a disk controller.

[0085] In the embodiment, the server can query the target data identifier consistent with the data identifier in the metadata table, and then determine the physical address corresponding to the target data identifier as the physical address of the target data in the disk, and obtain the physical address.

[0086] At step 402, whether the target data exists in the target cache space according to the physical address is queried.

[0087] The target cache space is used to store the deduplicated data and the persistent data in the disk.

[0088] It should be noted that when the target data does not exist in the multi-level cache space, the target data needs to be queried from the disk. However, the access speed of the server accessing the disk is much lower than that of the cache. Therefore, a cache space, i.e., the target cache space, can be allocated in the memory to store the core data of high-frequency access in the disk, i.e., the deduplicated data and the persistent data. The deduplicated data is the shared data after deduplication, and the persistent data is the key data that needs to be retained for a long time. It can be understood that, since the data amount of the deduplicated data is less than that of the persistent data, in order to improve the access efficiency, the server can first query whether the target data exists in the deduplicated data in the target cache space, and then query whether the target data exists in the persistent data in the target cache space when the target data does not exist in the deduplicated data.

[0089] The deduplicated data is data obtained by identifying and deleting the duplicate data blocks in the storage system, and only retaining the unique data copy and the reference of the copy. The deduplicated data can be completely cached in the target cache space to improve the access speed of the data. Optionally, the deduplicated data can adopt a radix tree as a storage form.

[0090] The persistent data refers to data that can be saved in a non-volatile storage medium after program termination, device power failure or system restart, and can be repeatedly accessed and used by subsequent operations. In order to improve the access speed of the data, the persistent data can be stored in the target cache space. Optionally, the persistent data can adopt a storage form of KV key-value pair combined with a hash table, wherein the K value in the KV key-value pair can be uniquely mapped with the physical address.

[0091] In the embodiment, the server can query whether the target physical address consistent with the physical address exists in the target cache space, and then determine whether the target data exists in the target cache space according to the query result.

[0092] Optionally, if the target physical address exists in the target cache space, it can be determined that the target data exists in the target cache space, and if the target physical address does not exist in the target cache space, it can be determined that the target data does not exist in the target cache space.

[0093] In this embodiment, the server first acquires the physical address of the target data in the disk from the preset metadata table according to the data identifier, so as to shorten the positioning time of the storage location, and then queries whether the target data exists in the target cache space of the deduplicated data and the persistent data in the storage disk according to the physical address, so that the target data can be matched in the cache directly. If the cache hits, the data access delay can be reduced from milliseconds to microseconds, so as to improve the access speed. If the cache misses, the disk can be read and written once according to the physical address, so as to improve the data query efficiency.

[0094] In one embodiment, in Figure 4 on the basis of the embodiment shown in Figure 5 The embodiment relates to the process that how the server queries whether the target data exists in the target cache space according to the physical address, as shown in Figure 5 Step 402 comprises Figure 5 the steps 501-503 shown in

[0095] Step 501, according to the physical address, queries whether the target physical address same as the physical address exists in the fingerprint index table in the deduplicated cache space included in the target cache space.

[0096] The fingerprint index table is used for storing the similar fingerprint information of similar data in the deduplicated processing process, and the mapping relationship between the data fingerprint and the data physical storage location can be saved in the fingerprint index table.

[0097] The target physical address refers to the physical address same as the physical address queried from the metadata table in the plurality of physical addresses included in the fingerprint index table.

[0098] In this embodiment, the server can compare the physical address with the plurality of physical addresses in the fingerprint index table respectively, and then determine whether the target physical address same as the physical address exists according to the comparison result.

[0099] Step 502, if the target physical address exists in the fingerprint index table, the target data is acquired from the deduplicated cache space.

[0100] In this embodiment, the server can determine the data corresponding to the target physical address in the deduplicated cache space as the target data and acquire the data if the target physical address same as the physical address exists in the fingerprint index table.

[0101] Step 503, if the target physical address does not exist in the fingerprint index table, whether the target physical address exists in the persistent cache space included in the target cache space is queried according to the physical address.

[0102] In the embodiment, the server can query whether the target physical address consistent with the physical address exists in the persistent cache space according to the processed physical address in the case that the target physical address same as the physical address does not exist in the fingerprint index table.

[0103] In the embodiment, the server queries whether the target physical address same as the physical address exists in the fingerprint index table in the deduplication cache space included in the target cache space according to the physical address, and then acquires the target data from the deduplication cache space in the case that the target physical address exists in the fingerprint index table, avoids repeatedly loading or reading data from the disk, and queries whether the target physical address exists in the persistent cache space included in the target cache space according to the physical address in the case that the target physical address does not exist in the fingerprint index table, avoids directly accessing the disk, which can fully combine the characteristics of the deduplication data and the persistent data, accurately match through the physical address, reduce the access times of the disk, reduce the dependence on the disk and the system load, and thus improve the data access performance.

[0104] In one embodiment, on the basis of the above embodiment, the embodiment relates to the process of how the server acquires the target data from the persistent cache space. The process includes:

[0105] Firstly, if the target physical address exists in the persistent cache space, the target data is acquired from the persistent cache space.

[0106] It should be noted that the target physical address in the persistent cache space is uniquely mapped with the K value.

[0107] In the embodiment, the server can generate the corresponding K value in the persistent cache space according to the target physical address in the case that the target physical address exists in the persistent cache space, and then determine the V value corresponding to the K value as the target data.

[0108] Secondly, if the target physical address does not exist in the persistent cache space, the target data is queried from the disk according to the physical address.

[0109] It should be noted that if the target physical address does not exist in the persistent cache space, it means that the target data is not included in the persistent data included in the persistent cache space.

[0110] In the embodiment, the server can directly query the target data from the disk according to the physical address in the case that the target physical address does not exist in the persistent cache space.

[0111] In the embodiment, the server acquires the target data from the persistent cache space when the target physical address exists in the persistent cache space, and queries the target data from the disk according to the physical address when the target physical address does not exist in the persistent cache space, so that a cooperative data storage architecture of cache acceleration and disk backup can be provided by querying the persistent cache space before querying the disk, the problem of data unavailability caused by loss of persistent cache data is avoided, when the cache hits, the access time is greatly shortened, when the cache misses, the disk is directly accessed through the physical address, the intermediate overhead is reduced, the number of disk read and write is reduced, and the storage performance is improved.

[0112] In one embodiment, based on the above embodiment, referring to Figure 6 The embodiment relates to a process of how the server writes the target data into the dynamic cache space after querying and acquiring the target data from the disk, as shown in Figure 6 The server can perform steps 601-603 to implement the process.

[0113] Step 601: Query the remaining storage capacity of the dynamic cache space.

[0114] The remaining storage capacity refers to the size of the unused storage resource after the total rated capacity in the dynamic cache space is subtracted from the capacity occupied by the actual data.

[0115] It should be noted that if the target data is read from the disk, in order to avoid the problem of slow access speed when accessing the data again, the data needs to be read from the disk again. However, if the remaining storage capacity of the dynamic cache is too low (for example, less than 10% of the total capacity), the cache hit rate will drop sharply, new data cannot enter the cache, the disk needs to be frequently read, and even cache overflow occurs, that is, data writing fails; if the remaining storage capacity of the dynamic cache is too high, the cache resource will be wasted, for example, the high-speed medium is not fully utilized, therefore, when writing data into the dynamic cache space, the remaining storage capacity of the dynamic cache space needs to be monitored.

[0116] In the embodiment, the server can collect the remaining storage capacity of the dynamic cache space in real time by calling a monitoring tool through a system interface.

[0117] Step 602: If the remaining storage capacity is greater than or equal to a preset storage capacity threshold, write the target data into the dynamic cache space.

[0118] The storage capacity threshold is a critical value to avoid the dynamic cache space being full.

[0119] In the embodiment, the server can determine that the data can be written into the dynamic cache space without affecting the data reading and data writing of the dynamic cache space when the remaining storage capacity is greater than or equal to the storage capacity threshold, and then write the target data into the dynamic cache space.

[0120] In step 603, if the remaining storage capacity is less than the storage capacity threshold, a garbage collection operation is performed on the dynamic cache space, and after the garbage collection operation is performed, the target data is written into the dynamic cache space.

[0121] The garbage collection operation is a mechanism for automatically identifying and cleaning invalid data (such as expired data, redundant data, and unreferenced data) and releasing the occupied cache space.

[0122] In the embodiment, the server can determine that the data cannot be written into the dynamic cache space when the remaining storage capacity is less than the storage capacity threshold, at which time the garbage collection operation can be called to delete the data in the dynamic cache space according to the heat information of the data access, and then the target data is written into the dynamic cache space after the garbage collection operation is performed.

[0123] In the embodiment, the server can write the target data into the dynamic cache space when the remaining storage capacity is greater than or equal to the preset storage capacity threshold by querying the remaining storage capacity of the dynamic cache space, and perform a garbage collection operation on the dynamic cache space when the remaining storage capacity is less than the storage capacity threshold, and then write the target data into the dynamic cache space after the garbage collection operation is performed. In this way, the data stored in the dynamic cache space can be dynamically adjusted by actively controlling the remaining storage capacity and performing the garbage collection operation, which avoids the dynamic cache space from being filled up, solves the problem of insufficient remaining storage capacity but existence of invalid data, thereby improving the reliability of the target data writing and improving the resource utilization.

[0124] In one embodiment, a data query method is provided for a server, as shown in Figure 7 The method includes the following steps:

[0125] In step 701, a data access request is received.

[0126] The data access request includes a data identifier of target data.

[0127] In step 702, whether the target data exists in the dynamic cache space is queried according to the data identifier.

[0128] The dynamic cache space is used for data storage according to data heat.

[0129] Step 703, if the target data exists in the dynamic cache space, obtaining the target data from the dynamic cache space.

[0130] Step 704, if the target data does not exist in the dynamic cache space, querying whether the target data exists in the front-end cache space according to the data identifier.

[0131] The front-end cache space is used to store front-end data.

[0132] Step 705, if the target data exists in the front-end cache space, obtaining the target data from the front-end cache space.

[0133] Step 706, if the target data does not exist in the front-end cache space, obtaining the physical address of the target data in the disk from a preset metadata table according to the data identifier.

[0134] Step 707, querying whether the target physical address exists in the fingerprint index table in the deduplication cache space included in the target cache space according to the physical address.

[0135] Step 708, if the target physical address exists in the fingerprint index table, obtaining the target data from the deduplication cache space.

[0136] Step 709, if the target physical address does not exist in the fingerprint index table, querying whether the target physical address exists in the persistent cache space included in the target cache space according to the physical address.

[0137] Step 7010, if the target physical address exists in the persistent cache space, obtaining the target data from the persistent cache space.

[0138] Step 7011, if the target physical address does not exist in the persistent cache space, querying the target data from the disk according to the physical address.

[0139] Step 7012, querying the remaining storage capacity of the dynamic cache space.

[0140] Step 7013, if the remaining storage capacity is greater than or equal to a preset storage capacity threshold, writing the target data into the dynamic cache space.

[0141] Step 7014, if the remaining storage capacity is less than the storage capacity threshold, performing a garbage collection operation on the dynamic cache space, and writing the target data into the dynamic cache space after performing the garbage collection operation.

[0142] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately executed with other steps or steps or stages in other steps.

[0143] Based on the same inventive concept, the embodiments of the present application also provide a data query device for implementing the above-mentioned data query method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more data query device embodiments provided below can refer to the limitations of the data query method in the above text, which will not be repeated here.

[0144] In one exemplary embodiment, as shown in Figure 8 a data query device is provided, comprising:

[0145] The receiving module 801 is configured to receive a data access request, wherein the data access request comprises a data identifier of target data.

[0146] The first query module 802 is configured to, in response to the data access request, query whether the target data exists in a plurality of hierarchical cache spaces according to the data identifier, wherein different hierarchical cache spaces correspond to different data access speeds.

[0147] The second query module 803 is configured to, if the target data does not exist in the plurality of hierarchical cache spaces, query the target data from a disk.

[0148] In one embodiment, the plurality of hierarchical cache spaces comprises a dynamic cache space and a front-end cache space, and the first query module 802 comprises:

[0149] The first query unit is configured to query whether the target data exists in the dynamic cache space according to the data identifier, wherein the dynamic cache space is configured to store data according to data hotness.

[0150] The first acquisition unit is configured to, if the target data exists in the dynamic cache space, acquire the target data from the dynamic cache space.

[0151] The second query unit is configured to query whether the target data exists in the front-end cache space according to the data identifier if the target data does not exist in the dynamic cache space, and the front-end cache space is configured to store front-end data.

[0152] In one of the embodiments, before querying the target data from the disk, the apparatus further comprises:

[0153] The first obtaining module is configured to obtain the physical address of the target data in the disk from the preset metadata table according to the data identifier.

[0154] The third query module is configured to query whether the target data exists in the target cache space according to the physical address, and the target cache space is configured to store the deduplicated data and the persistent data in the disk.

[0155] In one of the embodiments, the third query module comprises:

[0156] The third query unit is configured to query whether the target physical address identical to the physical address exists in the fingerprint index table in the deduplicated cache space included in the target cache space according to the physical address.

[0157] The second obtaining unit is configured to obtain the target data from the deduplicated cache space if the target physical address exists in the fingerprint index table.

[0158] The fourth query unit is configured to query whether the target physical address exists in the persistent cache space included in the target cache space according to the physical address if the target physical address does not exist in the fingerprint index table.

[0159] In one of the embodiments, the apparatus further comprises:

[0160] The second obtaining module is configured to obtain the target data from the persistent cache space if the target physical address exists in the persistent cache space.

[0161] The second query module 803 comprises:

[0162] The fifth query unit is configured to query the target data from the disk according to the physical address if the target physical address does not exist in the persistent cache space.

[0163] In one of the embodiments, after querying and obtaining the target data from the disk, the apparatus further comprises:

[0164] The fourth query module is configured to query the remaining storage capacity of the dynamic cache space.

[0165] The first writing module is configured to write the target data into the dynamic cache space if the remaining storage capacity is greater than or equal to the preset storage capacity threshold.

[0166] The second writing module is configured to perform a garbage collection operation on the dynamic cache space if the remaining storage capacity is less than the storage capacity threshold, and write the target data into the dynamic cache space after the garbage collection operation.

[0167] The modules in the data query device can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in the server in hardware form, or stored in a memory in the server in software form, so as to be called and executed by the processor.

[0168] In an exemplary embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in Figure 9 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store data to be queried. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with terminals outside through a network connection. The computer program is executed by the processor to implement a data query method.

[0169] Those skilled in the art can understand that Figure 9 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0170] In an exemplary embodiment, a computer device is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the following steps:

[0171] receiving a data access request, the data access request including a data identifier of target data;

[0172] In response to the data access request, querying whether the target data exists in a plurality of hierarchical cache spaces according to the data identifier, different hierarchical cache spaces corresponding to different data access speeds;

[0173] If the target data does not exist in the plurality of hierarchical cache spaces, the target data is queried from the disk.

[0174] In one embodiment, the processor implements the following steps when executing the computer program:

[0175] If the target data exists in the dynamic cache space, the target data is obtained from the dynamic cache space;

[0176] If the target data does not exist in the dynamic cache space, whether the target data exists in the front-end cache space is queried according to the data identifier, the front-end cache space being used to store front-end data.

[0177] In one embodiment, the processor implements the following steps when executing the computer program:

[0178] According to the data identifier, a physical address of the target data in the disk is obtained from a preset metadata table;

[0179] According to the physical address, whether the target data exists in the target cache space is queried, the target cache space being used to store the deduplicated data and the persistent data in the disk.

[0180] In one embodiment, the processor implements the following steps when executing the computer program:

[0181] According to the physical address, whether a target physical address same as the physical address exists in a fingerprint index table in a deduplicated cache space included in the target cache space is queried;

[0182] If the target physical address exists in the fingerprint index table, the target data is obtained from the deduplicated cache space;

[0183] If the target physical address does not exist in the fingerprint index table, whether the target physical address exists in a persistent cache space included in the target cache space is queried according to the physical address.

[0184] In one embodiment, the processor implements the following steps when executing the computer program:

[0185] If the target physical address exists in the persistent cache space, the target data is obtained from the persistent cache space;

[0186] The target data is queried from the disk, including:

[0187] If the target physical address does not exist in the persistent cache space, the target data is queried from the disk according to the physical address.

[0188] In one embodiment, the processor implements the following steps when executing the computer program:

[0189] querying a remaining storage capacity of the dynamic cache space;

[0190] if the remaining storage capacity is greater than or equal to a preset storage capacity threshold, writing the target data into the dynamic cache space;

[0191] if the remaining storage capacity is less than the storage capacity threshold, performing a garbage collection operation on the dynamic cache space, and writing the target data into the dynamic cache space after performing the garbage collection operation.

[0192] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the following steps:

[0193] receiving a data access request, the data access request including a data identifier of target data;

[0194] in response to the data access request, querying whether the target data exists in a plurality of hierarchical cache spaces according to the data identifier, different hierarchical cache spaces corresponding to different data access speeds;

[0195] if the target data does not exist in the plurality of hierarchical cache spaces, querying the target data from a disk.

[0196] In one embodiment, the computer program is executed by the processor to implement the following steps:

[0197] if the target data exists in the dynamic cache space, obtaining the target data from the dynamic cache space;

[0198] if the target data does not exist in the dynamic cache space, querying whether the target data exists in a front-end cache space according to the data identifier, the front-end cache space being used to store front-end data.

[0199] In one embodiment, the computer program is executed by the processor to implement the following steps:

[0200] obtaining a physical address of the target data in the disk from a preset metadata table according to the data identifier;

[0201] querying whether the target data exists in a target cache space according to the physical address, the target cache space being used to store deduplicated data and persistent data in the disk.

[0202] In one embodiment, the computer program is executed by the processor to implement the following steps:

[0203] querying whether a target physical address identical to the physical address exists in a fingerprint index table in a deduplicated cache space included in the target cache space according to the physical address;

[0204] if the target physical address exists in the fingerprint index table, obtaining the target data from the re-deletion cache space;

[0205] if the target physical address does not exist in the fingerprint index table, querying whether the target physical address exists in a persistent cache space included in the target cache space according to the physical address.

[0206] In one of the embodiments, the computer program, when executed by the processor, specifically implements the following steps:

[0207] if the target physical address exists in the persistent cache space, obtaining the target data from the persistent cache space;

[0208] querying the target data from the disk, comprising:

[0209] if the target physical address does not exist in the persistent cache space, querying the target data from the disk according to the physical address.

[0210] In one of the embodiments, the computer program, when executed by the processor, specifically implements the following steps:

[0211] querying the remaining storage capacity of the dynamic cache space;

[0212] if the remaining storage capacity is greater than or equal to the preset storage capacity threshold, writing the target data into the dynamic cache space;

[0213] if the remaining storage capacity is less than the storage capacity threshold, performing a garbage collection operation on the dynamic cache space, and writing the target data into the dynamic cache space after performing the garbage collection operation.

[0214] In one of the embodiments, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the following steps:

[0215] receiving a data access request, the data access request comprising a data identifier of target data;

[0216] in response to the data access request, querying whether the target data exists in a plurality of hierarchical cache spaces according to the data identifier, different hierarchical cache spaces corresponding to different data access speeds;

[0217] if the target data does not exist in the plurality of hierarchical cache spaces, querying the target data from the disk.

[0218] In one of the embodiments, the computer program, when executed by the processor, specifically implements the following steps:

[0219] if the target data exists in the dynamic cache space, obtaining the target data from the dynamic cache space;

[0220] If the target data does not exist in the dynamic cache space, it is determined whether the target data exists in a front-end cache space according to the data identifier, the front-end cache space being used to store front-end data.

[0221] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0222] According to the data identifier, a physical address of the target data in the disk is obtained from a preset metadata table;

[0223] According to the physical address, it is determined whether the target data exists in a target cache space, the target cache space being used to store deduplicated data and persistent data in the disk.

[0224] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0225] According to the physical address, it is determined whether a target physical address identical to the physical address exists in a fingerprint index table in a deduplicated cache space included in the target cache space;

[0226] If the target physical address exists in the fingerprint index table, the target data is obtained from the deduplicated cache space;

[0227] If the target physical address does not exist in the fingerprint index table, it is determined whether the target physical address exists in a persistent cache space included in the target cache space according to the physical address.

[0228] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0229] If the target physical address exists in the persistent cache space, the target data is obtained from the persistent cache space;

[0230] The target data is queried from the disk, including:

[0231] If the target physical address does not exist in the persistent cache space, the target data is queried from the disk according to the physical address.

[0232] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0233] The remaining storage capacity of the dynamic cache space is queried;

[0234] If the remaining storage capacity is greater than or equal to a preset storage capacity threshold, the target data is written into the dynamic cache space;

[0235] If the remaining storage capacity is less than the storage capacity threshold, a garbage collection operation is performed on the dynamic cache space, and after the garbage collection operation is performed, the target data is written into the dynamic cache space.

[0236] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, databases or other media used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0237] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present disclosure.

[0238] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the patent scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A data query method, characterized by, The method comprises: receiving a data access request, wherein the data access request comprises a data identifier of target data; in response to the data access request, querying whether the target data exists in a plurality of hierarchical cache spaces according to the data identifier, wherein different hierarchical cache spaces correspond to different data access speeds; if the target data does not exist in the plurality of hierarchical cache spaces, querying the target data from a disk.

2. The method of claim 1, wherein, The plurality of hierarchical cache spaces comprises a dynamic cache space and a front-end cache space, and the querying whether the target data exists in the plurality of hierarchical cache spaces according to the data identifier comprises: querying whether the target data exists in the dynamic cache space according to the data identifier, wherein the dynamic cache space is used for storing data according to data hotness; if the target data exists in the dynamic cache space, obtaining the target data from the dynamic cache space; if the target data does not exist in the dynamic cache space, querying whether the target data exists in the front-end cache space according to the data identifier, wherein the front-end cache space is used for storing front-end data.

3. The method according to claim 1 or 2, characterized in that, Before the querying the target data from the disk, the method further comprises: obtaining a physical address of the target data in the disk from a preset metadata table according to the data identifier; querying whether the target data exists in a target cache space according to the physical address, wherein the target cache space is used for storing deduplicated data and persistent data in the disk.

4. The method of claim 3, wherein, The querying whether the target data exists in the target cache space according to the physical address comprises: querying whether a target physical address same as the physical address exists in a fingerprint index table in a deduplicated cache space included in the target cache space according to the physical address; if the target physical address exists in the fingerprint index table, obtaining the target data from the deduplicated cache space; if the target physical address does not exist in the fingerprint index table, querying whether the target physical address exists in a persistent cache space included in the target cache space according to the physical address.

5. The method of claim 4, wherein, The method further comprises: if the target physical address exists in the persistent cache space, obtaining the target data from the persistent cache space; The querying the target data from the disk comprises: if the target physical address does not exist in the persistent cache space, querying the target data from the disk according to the physical address.

6. The method of claim 2, wherein, After the querying and obtaining the target data from the disk, the method further comprises: querying a remaining storage capacity of the dynamic cache space; if the remaining storage capacity is greater than or equal to a preset storage capacity threshold, writing the target data into the dynamic cache space; if the remaining storage capacity is less than the storage capacity threshold, performing a garbage collection operation on the dynamic cache space, and writing the target data into the dynamic cache space after performing the garbage collection operation.

7. A data query apparatus, characterized by comprising: The device comprises: A receiving module is configured to receive a data access request, wherein the data access request comprises a data identifier of target data; A first querying module is configured to, in response to the data access request, query whether the target data exists in a plurality of hierarchical cache spaces according to the data identifier, wherein different hierarchical cache spaces correspond to different data access speeds; A second querying module is configured to, if the target data does not exist in the plurality of hierarchical cache spaces, query the target data from a disk.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images