Data query method and device of storage system, equipment and medium

Abstract

The invention discloses a data query method and device for a storage system, equipment and a medium. The method comprises the steps of obtaining an address interval of to-be-queried data; determining at least one target partition corresponding to the address interval according to pre-stored data partition information of the storage system; when data query is performed on a first partition in the at least one target partition, determining a next to-be-queried second partition corresponding to the first partition according to a pre-stored transition probability matrix, and performing data pre-reading on the second partition to obtain at least one query result, elements in the transition probability matrix are used for representing pre-reading probabilities among data in the storage system; and summarizing the at least one query result to obtain a target query result corresponding to the to-be-queried data. The range query performance is obviously optimized and the random query delay is reduced by indexing at least one partition, and in addition, the data pre-reading is realized through the transition probability matrix, so that the query efficiency can be effectively improved and the access waiting time is reduced.

Description

Technical Field

[0001] This application relates to the technical field of storage systems, and in particular, to a data query method, apparatus, device, and medium for a storage system. Background Art

[0002] A large amount of data is stored in the storage system. During operation, data will continue to be written at any time, or the existing data will be modified, or data will be queried and read from the storage system. Frequent access to the storage system may result in a long access waiting time due to the limitation of the global lock, and traditional query methods may have a certain data transmission delay in the storage scenario. Summary of the Invention

[0003] This application provides a data query method, apparatus, device, and medium for a storage system, so as to at least solve the problem that frequent access to the storage system may cause a long access waiting time due to the limitation of the global lock, and traditional query methods may have a certain data transmission delay in the storage scenario.

[0004] This application provides a data query method for a storage system, including: obtaining an address range of data to be queried; According to the pre-stored data partition information of the storage system, determining at least one target partition corresponding to the address range, where the data partition information is obtained by pre-dividing the data of the storage system according to the logical block address range; When querying data in the first partition of at least one target partition, according to the pre-stored transition probability matrix, determining a second partition to be queried next corresponding to the first partition, and performing data prefetching on the second partition to obtain at least one query result, where the elements in the transition probability matrix are used to represent the prefetching probability between each data in the storage system; Summarizing at least one query result to obtain a target query result corresponding to the data to be queried.

[0005] This application further provides a data query apparatus for a storage system, including: an obtaining module, configured to obtain an address range of data to be queried; A processing module, configured to determine at least one target partition corresponding to the address range according to the pre-stored data partition information of the storage system, where the data partition information is obtained by pre-dividing the data of the storage system according to the logical block address range; The processing module is further configured to, when querying data in the first partition of at least one target partition, determine a second partition to be queried next corresponding to the first partition according to the pre-stored transition probability matrix, and perform data prefetching on the second partition to obtain at least one query result, where the elements in the transition probability matrix are used to represent the prefetching probability between each data in the storage system; The processing module is further configured to summarize at least one query result to obtain a target query result corresponding to the data to be queried.

[0006] The present application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of the data query method of any one of the above storage systems when executing the computer program.

[0007] The present application also provides a computer-readable storage medium, in which a computer program is stored. Wherein, when the computer program is executed by a processor, the steps of the data query method of any one of the above storage systems are implemented.

[0008] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the data query method of any one of the above storage systems are implemented.

[0009] Through the present application, an address range of the data to be queried is obtained; at least one target partition corresponding to the address range is determined according to the pre-stored data partition information of the storage system, and the data partition information is obtained by previously dividing the data of the storage system according to the logical block address range; when querying data in the first partition among at least one target partition, according to the pre-stored transition probability matrix, the second partition to be queried next corresponding to the first partition is determined, and data prefetching is performed on the second partition to obtain at least one query result, and the elements in the transition probability matrix are used to represent the prefetching probability between each data in the storage system; at least one query result is summarized to obtain a target query result corresponding to the data to be queried. In this solution, by indexing at least one partition respectively, the range query performance is significantly optimized, and the random query latency is reduced. In addition, data prefetching is realized through the transition probability matrix, which can effectively improve the query efficiency and reduce the access waiting time. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0011] Figure 1 Flow chart of a data query method for a storage system provided by an embodiment of the present application Figure 1 ; Figure 2 Flow chart of a data query method for a storage system provided by an embodiment of the present application Figure 2 ; Figure 3Structural diagram of a data query device for a storage system provided by an embodiment of the present application; Figure 4 Structural diagram of an electronic device provided by an embodiment of the present application. Specific implementation manners

[0012] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0013] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0014] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0015] Modern storage systems may mainly face three core challenges: (1) Lock contention under high concurrency: Traditional indexes (such as B+ trees, LSM-Trees) suffer a significant performance drop due to global locks in distributed and centralized scenarios.

[0016] (2) Adaptability to mixed workloads: Random writes (such as database transactions) and sequential reads (such as data analysis) require different optimization strategies, and it is difficult for existing systems to balance both.

[0017] (3) Memory and storage wall: The access latency of DRAM (about 100 ns) and the latency of NVMe SSD (about 10 μs) differ by two orders of magnitude, and an efficient caching and prefetching mechanism is required.

[0018] Industry pain points: (1) Ceph: It implements distributed storage using the Controlled Replication Under Scalable Hashing (CRUSH) algorithm. However, its metadata management depends on a global Map, and its random write throughput is lower than 500K IOPS (4K blocks).

[0019] (2) Storage Performance Development Kit (SPDK): It improves performance through user-space polling, but lacks intelligent indexing, and the range query efficiency is only 30% of the point query.

[0020] (3) Aerospike: It uses a hybrid index (in-memory hash + disk B+ tree), but the memory fragmentation rate is as high as 15% - 20%.

[0021] It should be noted that the B+ tree refers to the Balanced Plus Tree, which is widely used in databases and file systems as an index structure to optimize the storage and retrieval efficiency of data. Compared with the B tree, the B+ tree stores all data records (or pointers to data records) at the leaf nodes, and the leaf nodes are connected to each other through pointers to form an ordered linked list, which makes range queries and sequential access more efficient.

[0022] It should be noted that the LSM-Tree refers to the Log-Structured Merge-Tree. The LSM-Tree optimizes the high-throughput write scenario through sequential writes and hierarchical merges, and is especially suitable for big data applications with more writes than reads (such as log storage, time series data). Utilizing the characteristic that the disk sequential write performance is much higher than the random write, all update operations (insert, delete, modify) are converted into append writes to avoid random I / O. The data is divided into a memory component (MemTable) and a disk component (SSTable), and redundant data is reduced through regular compaction to improve storage efficiency. The write operation only involves appending to the memory and does not require immediate disk updates, making it suitable for high-concurrency scenarios.

[0023] It should be noted that a global lock is a mechanism used to control concurrent access to shared resources in a database or distributed system. It ensures data consistency and operation atomicity by locking the entire system or critical resources. A global lock is a coarse-grained locking mechanism that locks the entire database, system, or critical resources to prevent other transactions or processes from accessing the protected resources concurrently. During the period when the global lock is in effect, any operation attempting to access the protected resources will be blocked until the lock is released. This mechanism is typically used in scenarios that require strict data consistency guarantees, such as database backups, system maintenance, or atomic updates of critical data structures.

[0024] It should be noted that DRAM can refer to Dynamic Random Access Memory. DRAM stores data in a row-column matrix form. When accessing, the target row needs to be activated first (Row Address Strobe, RAS), and then the specific cell is located through Column Address Strobe (CAS). This process involves the reading and amplification of capacitor charges, resulting in row activation latency (tRCD) and column access latency (tCAS). DRAM needs to be refreshed regularly to maintain data integrity (dynamic characteristics). The refresh operation occupies the memory bus, causing other access requests to queue. The process technology level of memory chips directly affects the charge and discharge speed. Advanced processes can shorten the capacitor charge and discharge time and reduce latency. In addition, attenuation and interference in signal transmission (such as bus length, noise) will also increase latency.

[0025] It should be noted that NVMe SSD can refer to Non-Volatile Memory Express SSD. The random read latency of NVMe SSD is usually between 20 and 80 microseconds (μs), and the random write latency is 50 to 150 microseconds. The total latency can be controlled within the range of 0.02 to 0.15 milliseconds (ms). This latency level has an order-of-magnitude advantage compared to traditional SATA SSDs and HDDs. For example, the latency of HDDs is usually as high as several milliseconds. The low latency of NVMe SSD makes it stand out in latency-sensitive scenarios such as database operations, virtual machine running, and real-time analysis.

[0026] To enable those skilled in the art of this technology to better understand the solution of this application, the following further detailed description of this application will be provided in conjunction with the accompanying drawings and specific implementation manners.

[0027] As Figure 1 shown, Figure 1 is a flowchart of the data query method for the storage system provided by the embodiment of this application. The method may include the following steps: 101. Obtain the address range of the data to be queried.

[0028] In an embodiment of the present application, the address range of the data to be queried can be expressed as [Lstart, Lend], which can refer to a range composed of a continuous logical block address.

[0029] It should be noted that the logical block address (LBA) is an addressing method used for disk storage devices (such as hard disks, solid-state drives, etc.). LBA is a method of mapping physical blocks on a disk to logical numbers, enabling the operating system to more conveniently access and manage data on the disk. LBA numbers all physical blocks on the disk in sequence, starting from 0, and each physical block has a unique logical address. The operating system accesses data on the disk through these logical addresses without caring about the actual physical location where the data is stored on the disk. Each LBA corresponds to a data block of a fixed size (such as 4KB), and in a hash index, LBA is usually used as a key (Key).

[0030] In some embodiments, the acquisition method of the address range of the data to be queried may at least include the following implementation methods: Implementation method one: User request.

[0031] When a user or application needs to read or write a continuous piece of data, a start address (L start ) and an end address (L end ) will be specified, thus forming a range. For example: When the user needs to read a part of the data in a file, and the file is stored between LBA 1000 and LBA 2000, then the address range is [1000, 2000].

[0032] Implementation method two: System task.

[0033] The system may need to execute some background tasks, such as data migration, backup, or garbage collection. These tasks usually involve a continuous LBA range. For example: When the system needs to migrate the data of a certain partition to another node, then a range [Lstart, Lend] will be specified to represent the data range to be migrated.

[0034] Implementation method three: Prefetch mechanism.

[0035] To improve performance, the system may predict the data that the user will need next based on the access pattern and prefetch a continuous LBA range in advance. For example: When the system predicts that the user will next read the data from LBA 1000 to LBA 1500, then the range is [1000, 1500].

[0036] 102. Determine at least one target partition corresponding to the address range according to the pre-stored data partition information of the storage system.

[0037] In the embodiments of the present application, a large amount of data is stored in the storage system. In order to better store and achieve faster data location, query, retrieval, etc., the storage system can be partitioned. The specific data partition information is obtained by pre-dividing the data of the storage system according to the logical block address range. After determining the address range of the data to be queried, it is necessary to determine the corresponding partition of this address range. The number of partitions can be at least one, that is, to determine at least one target partition covered by this address range.

[0038] In some embodiments, the system needs to know start , L end range of data is stored in which partitions in order to correctly read or write data. For example: if the interval [1000, 2000] is distributed in partitions S1, S2, and S3, then the system needs to read data from these partitions respectively. In addition, the query interval can help the system understand the uniformity of data distribution, so as to perform load balancing. For example: if the data volume of a certain partition is too large, the system can migrate some data to other partitions to avoid hot spot problems. At the same time, the query interval can help the system optimize data access performance. For example: if start , L end spans multiple partitions, the system can minimize cross-partition access and improve performance; if the interval start , L end is distributed in multiple partitions, the system can pre-read data from these partitions in advance to reduce latency. Also, when migrating data or expanding the system, the query interval can help the system determine the data range to be migrated. For example: if the system needs to migrate the data of a certain partition to a new node, then it is necessary to query the distribution of all intervals in this partition. Finally, when a system failure occurs, the query interval can help the system quickly locate the affected data range and perform recovery. For example: if a certain partition fails, the system can query the distribution of all intervals in this partition and recover the data from the backup.

[0039] 103. When querying data in the first partition among at least one target partition, according to the pre-stored transition probability matrix, determine the next second partition to be queried corresponding to the first partition, and perform data pre-reading on the second partition to obtain at least one query result.

[0040] In an embodiment of the present application, after determining at least one target partition corresponding to the address range, data query needs to be performed in the at least one target partition. During the data query process, the B+ tree structures in each target partition can be traversed in parallel, that is, the data in each target partition in at least one target partition is traversed simultaneously, so as to obtain at least one query result. The at least one query result can correspond to the at least one target partition, that is, each target partition can obtain a query result.

[0041] It should be noted that when performing data query on each target partition, in order to ensure the efficiency of data access query, a prefetch mechanism can be adopted. Prefetch can be simply understood as predicting which partition data will be queried next when querying the data in the current partition and prefetching it in advance. That is, when performing data query on the first partition, the second partition that may be queried next can be predicted for data prefetching. In this way, if the prediction is correct, there is no need to wait and the data can be directly queried. In order to improve the accuracy of data prefetch prediction, the partition can be specifically determined through a pre-stored transition probability matrix. The elements in the transition probability matrix can be used to represent the prefetch probability between various data in the storage system, that is, the transition probability matrix stores the prefetch probability between the first partition and other partitions. When performing data query on the first partition, the second partition can be determined through the prefetch probability between the first partition and other partitions and prefetch it. Of course, when querying other partitions, the steps are the same, and the data of the next partition that may be queried will be prefetched in advance through the prefetch mechanism, so as to obtain at least one query result, and the query result can correspond to the partition one by one.

[0042] 104. Aggregate at least one query result to obtain a target query result corresponding to the data to be queried.

[0043] In an embodiment of the present application, after querying each of the at least one target partitions to obtain at least one query result, the at least one query result can be aggregated to obtain the final target query result corresponding to the data to be queried.

[0044] Exemplarily, the process of the above data query can be transformed into a scenario of finding books in a library. Specifically, there are many bookshelves (partitions) in the library, and there are many books (data) on each bookshelf. Each book has a number (LBA), and it can be quickly found through this number. Now, a batch of books with consecutive numbers, such as books numbered from 1000 to 2000, need to be found. These books may be distributed on different bookshelves, so it is necessary to find books on each relevant bookshelf and then organize the found books together to form the final result.

[0045] In the specific process of finding books, several assistants (parallel tasks) can be sent to different bookshelves to find books (parallel traversal of partitions) simultaneously. For example, assistant A goes to bookshelf 1, assistant B goes to bookshelf 2, and assistant C goes to bookshelf 3. Each assistant has a map (B+ tree) and can quickly find books (data) numbered between 1000 and 2000. For example, assistant A finds books {D1, D2}, assistant B finds books {D3, D4}, and assistant C finds book {D5}. Finally, after all assistants have found books (data), the books (data) found by all assistants can be aggregated together to form the final result (target query result), that is, a stack of books numbered from 1000 to 2000 is obtained: {D1, D2, D3, D4, D5}.

[0046] In the embodiment of the present application, by indexing at least one partition respectively, the range query performance is significantly optimized, and the random query latency is reduced. In addition, data prefetching is realized through the transition probability matrix, which can effectively improve the query efficiency and reduce the access waiting time.

[0047] As Figure 2 shown, Figure 2 Another flowchart of the data query method for the storage system provided by the embodiment of the present application is shown. The method may include the following steps: 201. Obtain the address range of the data to be queried.

[0048] In the embodiment of the present application, for the description of step 201, please refer to the detailed description of step 101 in the above embodiment, and the embodiment of the present application will not repeat it.

[0049] 202. Obtain the data to be written.

[0050] In the embodiment of the present application, before operations such as querying and reading data, these data need to be written into the storage system first. This writing can be understood as adding new data or modifying the existing stored data, etc.

[0051] 203. Map the data to be written according to a preset hash function, and store the data to be written into the corresponding partition to obtain data partition information.

[0052] In the embodiments of the present application, since there are multiple partitions in the storage system, when writing data, it is necessary to determine which specific partition each data needs to be written to. At this time, it is necessary to map the data to be written to determine the corresponding partition of the data to be written, so as to store the data to be written in the corresponding partition. Since the number of the data to be written may be at least one, the corresponding partition may also be at least one partition. Each data will correspond to a partition to be stored. After all the storage is completed, the data partition information can be obtained. The data partition information can indicate which data is stored in each partition of the storage system.

[0053] It should be noted that when mapping the data to be written, it can be implemented by a preset hash function. The preset hash function can be an improved Fibonacci Hashing function, and the improved Fibonacci Hashing function can be expressed as:

[0054] Among them, k can represent the original key value (such as the LBA address), which is input by an external (user); N can represent the total number of partitions, is the golden ratio, which is a fixed constant and can be used to reduce hash conflicts.

[0055] In some embodiments, h(k) represents the calculation result of the hash function H on the input key k. Specifically, h(k) is a function that maps the key k to a hash value. This hash value is usually used to determine the storage location of the data in the hash table. The main function of h(k) is to map the key k to a partition number, so as to determine which partition the data should be stored in.

[0056] In some embodiments, during data writing, Compare-And-Swap (CAS) insertion can be performed. CAS insertion is a commonly used lock-free concurrency control mechanism mainly used to implement thread-safe atomic operations. When referring to "CAS insertion", it generally means that in a concurrent environment, the CAS mechanism is used to safely insert elements into data structures (such as linked lists, trees, hash tables, etc.). The thread first reads the current value of the target memory location (or a certain node in the data structure), and the thread attempts to update the value of the target memory location to a new value, but this operation is conditional: the update will only succeed when the current value of the target memory location is the same as the value read by the thread before. This is usually achieved through a special CPU instruction (such as CMPXCHG in the x86 architecture). If the update is successful, it means that no other thread has modified the value of the target memory location during this period, and the insertion operation is completed; if the update fails, it means that some other thread has modified the value of the target memory location during this period, and the current thread needs to reread the current value and may retry the insertion operation (this is usually achieved through a loop until the insertion is successful or other termination conditions are met).

[0057] In some embodiments, the storage system has pre-implemented partitioning. Assuming the total LBA range of the storage space is [0, L max , and the number of partitions is N, then the LBA subspace managed by partition S i can be expressed as:

[0058] where L max can represent the maximum LBA address, N can represent the total number of partitions, initialized according to the cluster scale (such as 1024 partitions), and i can represent the partition index (from 0 to N - 1).

[0059] In some embodiments, when writing data, due to the limited number of partitions, there may be partition conflicts between data. Specifically, the data to be written is mapped according to a preset hash function, and the data to be written is stored in the corresponding partition to obtain data partition information, which may include: mapping the data to be written according to the preset hash function to obtain a mapping result; when the mapping result indicates that there are partition conflicts among at least two conflicting data, merging the at least two conflicting data to obtain a data merging interval; updating the version number according to the data merging interval and storing the data merging interval to obtain data partition information.

[0060] It should be noted that data conflicts may be reflected in partition mapping conflicts, in-bucket conflicts, linked list or tree structure degradation, etc. Among them, partition mapping conflict means that different keys are mapped to the same partition. Suppose the hash function h(k) maps keys to partition numbers, and the number of partitions N = 1024. If there are two keys k1 = 123456 and k2 = 654321, and h(k1) = h(k2) = 512, then k1 and k2 are mapped to the same partition S512, which is the partition mapping conflict. In-bucket conflict means that within the same partition, different keys are mapped to the same bucket (Bucket). Suppose there are multiple buckets in partition Si, and each bucket can store a certain number of keys. If two keys k1 and k2 are mapped to the same partition Si and further mapped to the same bucket, this is the in-bucket conflict. Linked list or tree structure degradation means that when a conflict occurs, the hash table may degenerate into a linked list or tree structure, resulting in a decrease in query efficiency. If the buckets in partition Si use a linked list to handle conflicts, when multiple keys are mapped to the same bucket, the length of the linked list will increase, and the query time complexity will degrade from O(1) to O(n).

[0061] When hash conflicts occur (i.e., different LBAs are mapped to the same partition or bucket), directly storing these LBAs will cause the following problems: Low storage efficiency: Each LBA needs to be stored separately, increasing the metadata overhead; Poor query performance: Multiple independent LBAs need to be traversed, increasing the query time.

[0062] By merging adjacent LBAs, the number of hash conflicts can be reduced. If two LBAs are adjacent (such as LBA1 = 1000 and LBA2 = 1001), they are very likely to belong to the same data block or consecutive data blocks. Merging them into one interval (such as [1000, 1001]) can reduce the number of hash conflicts because only one interval needs to be stored instead of multiple independent LBAs. After merging adjacent LBAs, multiple LBAs can be stored as one interval, reducing the overhead of metadata (such as pointers, index entries). For example, storing one interval [1000, 1001] only requires one index entry, while storing two independent LBAs requires two index entries. After merging adjacent LBAs, the entire interval can be quickly located during query without traversing multiple independent LBAs. For example, when querying LBA = 1000, the interval [1000, 1001] can be directly located instead of querying LBA1 = 1000 and LBA2 = 1001 separately. In practical applications, many access patterns are sequential (such as continuously reading multiple data blocks), and merging adjacent LBAs can better adapt to this sequential access pattern, reducing the overhead of prefetching and caching.

[0063] Therefore, if there are partition conflicts among at least two conflicting data, the conflicting data can be merged before storage. At the same time, it is also necessary to synchronously update the partition version number to avoid system errors.

[0064] In some embodiments, data conflicts do not necessarily occur during writing. The conflict probability is determined by the number of partitions, the size of the buckets within a partition, and the number of keys. Specifically, the conflict probability can be expressed as:

[0065] where N can represent the total number of partitions, B can represent the number of hash buckets managed by each partition, with a default value of 64, and K can represent the current total number of keys. When N = 1024, B = 64, and K = 10 8 then, P collision <0.05%.

[0066] In some embodiments, to reduce conflicts, the number of drawers (partitions) (N) can be increased. The more drawers there are, the larger the space for distributing items (data), and the lower the conflict probability. For example, increasing the number of drawers from 512 to 1024 will significantly reduce the conflict probability. The hash function can also be optimized. The role of the hash function is to evenly distribute items (data) into drawers (partitions). Using a better hash function (such as a hash function based on the golden ratio) can make the item distribution more uniform and reduce conflicts. The number of items (data) (K) can also be controlled. The fewer the number of items (data), the lower the conflict probability. For example, when K = 108, the conflict probability is much lower than when K = 1000. The hash space (B) can also be increased. The larger the hash space, the more dispersed the positions where items (data) are mapped, and the lower the conflict probability. For example, B = 64 can reduce conflicts better than B = 32.

[0067] In some embodiments, the lower the conflict probability, the higher the performance and efficiency of the system. Specifically, when the conflict probability is low, data query is faster because there is no need to handle crowded drawers; when the conflict probability is low, the item distribution is more uniform and the system load is more balanced; when the conflict probability is low, the resource utilization rate is higher and no additional resources are required to handle conflicts; when the conflict probability is low, the system is more stable and less likely to have performance bottlenecks.

[0068] Through this solution, if there are conflicts between data, the data can be merged before storage, which can reduce the number of hash conflicts, optimize the storage structure by reducing metadata, and can achieve fast positioning of intervals during query to improve query efficiency.

[0069] In some embodiments, during the process of data writing, in order to save space and also reduce the data size to improve the transmission efficiency, the data can be compressed. Specifically, the data to be written is stored in the corresponding partition to obtain data partition information, which may include: detecting the data transmission delay situation in real time; determining the data compression level according to the data transmission delay situation; and storing the data to be written in the corresponding partition after compression according to the data compression level to obtain data partition information.

[0070] It should be noted that the compression level can be dynamically adjusted according to the system load. Specifically, the compression level can be expressed as:

[0071] Among them, the SLA maximum allowable delay refers to the maximum delay (ms) allowed by the service level agreement, which can be custom-configured by the user. The current IO delay can be obtained by real-time monitoring or can be collected according to a certain sampling period (such as 10s). The calculated compression level can be custom-set. For example, there are a total of 9 levels (levels 1 to 9), with level 1 being the largest and level 9 being the smallest, etc.

[0072] In some embodiments, during data compression, if some data blocks are small, some adjacent small data blocks can be merged first and then compressed. For example, multiple 4KB data are merged into 1MB data and then compressed. After compression, it is required to satisfy:

[0073] Among them, B merged can represent the size of the merged data block, and B compressed can represent the size of the compressed data block. 4KB can refer to the size of the smallest storage unit, which is a fixed constant. When B merged = 1MB, the compression ratio is increased to 3.2 times.

[0074] Through this solution, the data can be compressed according to the data transmission delay situation, which can reduce the storage space required for the data, improve the utilization rate of the storage system, enable more data to be stored, and at the same time ensure the efficiency of data transmission.

[0075] 204. Save the data partition information.

[0076] In the embodiments of the present application, after obtaining the data partition information, the data partition information can be stored for subsequent calculations.

[0077] Through this solution, partitioned writing of data can be achieved. By introducing a hash function and a lock-free design, data conflict situations can be effectively avoided. At the same time, the system writing ability can be efficiently expanded in a distributed scenario.

[0078] 205. Determine at least one target partition corresponding to the address range according to the pre-stored data partition information of the storage system.

[0079] In the embodiments of the present application, for the description of step 205, please refer to the detailed description of step 102 in the above embodiments, and the embodiments of the present application will not be elaborated herein.

[0080] 206. Obtain the historical access sequence.

[0081] In the embodiments of the present application, the historical access sequence can be used to represent the access order of each piece of data during the historical data query process. For example, if the access sequence is [LBA1, LBA2, LBA3, LBA2, LBA3, LBA1], it can represent that the data corresponding to LBA1 is accessed first, then the data corresponding to LBA2 is accessed, then the data corresponding to LBA3 is accessed, then the data corresponding to LBA2 is accessed, then the data corresponding to LBA3 is accessed, and finally the data corresponding to LBA1 is accessed.

[0082] In some embodiments, the access sequence can be the access sequence between data or the access sequence between partitions. The historical access sequence is used to calculate the transition probability matrix for subsequent prefetching. That is to say, during prefetching, prefetching can be performed between data and data, or between partitions and partitions, and the logical steps are the same.

[0083] 207. According to the historical access sequence, count the number of transfers between the first data and the second data.

[0084] In the embodiments of the present application, both the first data and the second data are any data in the storage system. That is to say, it is necessary to count the number of transfers between any two pieces of data, and there is a direction difference in the transfer between data. That is, the transfer from the first data to the second data and the transfer from the second data to the first data are different concepts, and the number of times needs to be counted separately.

[0085] Exemplarily, if the access sequence is [LBA1, LBA2, LBA3, LBA2, LBA3, LBA1], it can be seen during counting that the number of times LBA1 transfers to LBA2 is 1, the number of times LBA1 transfers to LBA3 is 0, the number of times LBA2 transfers to LBA1 is 0, the number of times LBA2 transfers to LBA3 is 2, the number of times LBA3 transfers to LBA1 is 1, and the number of times LBA3 transfers to LBA2 is 1.

[0086] 208. Determine the prefetch probability between the first data and the second data according to the number of transfers.

[0087] In the embodiments of the present application, the prefetch probability can be expressed as the ratio between the number of transfers and the total number of times. The prefetch probability can be expressed as:

[0088] where P ij can represent the prefetch probability from state S i to S j . S i represents the current state (the currently accessed LBA or LBA group), and S j represents the next state (the next possible LBA or LBA group to be accessed). P ij reflects the prefetch rule in the LBA access pattern, is used to construct the state transition matrix, and optimize the prefetch strategy.

[0089] Exemplarily, if the access sequence is [LBA1, LBA2, LBA3, LBA2, LBA3, LBA1], then it can be seen during statistics that the number of times LBA1 transfers to LBA2 is 1, the number of times LBA1 transfers to LBA3 is 0, the number of times LBA2 transfers to LBA1 is 0, the number of times LBA2 transfers to LBA3 is 2, the number of times LBA3 transfers to LBA1 is 1, and the number of times LBA3 transfers to LBA2 is 1. After calculation, P 12 = 1 (always access LBA2 after LBA1), P 23 = 1 (always access LBA3 after LBA2), P 32 = 0.5 (there is a 50% probability of accessing LBA2 after LBA3), P 31 = 0.5 (there is a 50% probability of accessing LBA1 after LBA3).

[0090] 209. Aggregate the prefetch probabilities to obtain the transition probability matrix.

[0091] In the embodiments of the present application, according to the prefetch probabilities calculated above, after combination, the transition probability matrix can be obtained.

[0092] In some embodiments, according to the historical access sequence and the current access data situation, the data within the partition can also be divided into queues, thereby forming a dual LRU queue model, which can specifically include a Hot queue and a Warm queue. Among them, the Hot queue can store high-frequency access data, accounting for about 20% of the capacity; the Warm queue can store low-frequency access but possibly prefetchable data, accounting for about 80% of the capacity.

[0093] In some embodiments, through the above queue division, the hit rate can be calculated, thereby adjusting the queue and the prefetch strategy. The hit rate can be expressed as:

[0094] where λ h and λ w can respectively represent the real-time access frequencies (times / second) of the Hot queue and the Warm queue, and C h and C w can respectively represent the capacities (number of entries) of the Hot queue and the Warm queue, and can be fixed constants; can represent the proportion of hot data requests, which can be obtained through access log analysis.

[0095] By calculating this hit rate, it can be analyzed whether the queue division is correct and whether the prefetch policy is appropriate, and the hit rate can be promoted to adjust the data queue.

[0096] 210. When querying data in the first partition, retrieve the corresponding prefetch probability between the first partition and at least one other partition from the transition probability matrix.

[0097] In the embodiments of the present application, when querying data in the first partition, in order to implement prefetching, the corresponding prefetch probability between the first partition and at least one other partition can be determined through the transition probability matrix; of course, in order to ensure the accuracy of prefetching, the corresponding prefetch probability between the first partition and all other partitions can be determined.

[0098] In some embodiments, during the data query process, it includes point query and range query.

[0099] Point query refers to querying a single data item, such as finding the corresponding value (Value) through a key (Key). In hash partitioning, data is distributed to multiple partitions, and each partition is located through a hash function. The time complexity of the hash function is O(1) because it directly locates to the partition by calculating the hash value and does not need to traverse. Therefore, the time complexity of point query is O(1). Suppose you want to query the data with the key "A": calculate the hash value H("A") = Si (partition number), and directly search for the data in partition Si. The whole process is O(1).

[0100] Range query refers to querying a continuous range of data. For example, querying all data where the query key is within the range [K1, K2]. In a range query, it is necessary to traverse the B+ tree structure to locate the data. The time complexity of the B+ tree is O(logn), where n is the total amount of data; the time complexity of partitioning is O(m). A range query may involve multiple partitions, and m is the number of partitions covered. It is necessary to query the data in these partitions separately. Therefore, the time complexity of a range query is O(logn + m). Suppose you want to query data where the key is within the range [1000, 2000]. Traverse the B+ tree to find all the data that meets the conditions. The time complexity of this step is O(logn). If this data is distributed in m partitions, it is necessary to query the data in these partitions separately. The time complexity of this step is O(m). Therefore, the total time complexity is O(logn + m).

[0101] By calculating the time complexity, the query performance of the system can be evaluated, the system design can be optimized, the query strategy can be guided, the scalability of the system can be predicted, and the performance of different systems can be compared.

[0102] 211. If the prefetch probability between the first partition and the second partition is greater than the preset probability threshold and greater than the prefetch probability between the first partition and other partitions, then perform data prefetch on the second partition to obtain at least one query result.

[0103] In the embodiments of the present application, when performing prefetch, it is necessary to determine the next partition to be queried. Therefore, the transition probability between the first partition and other partitions can be compared. The preset probability threshold can be a custom prefetch trigger threshold. That is to say, as long as there is a probability greater than the preset probability threshold among the prefetch probabilities between the first partition and other partitions, then the next prefetch can be performed for the first partition, and then the maximum probability is selected from the probabilities greater than the preset probability threshold, and the corresponding partition is the second partition to be queried next.

[0104] In some embodiments, the conditions for prefetch can be expressed as:

[0105] Among them, can be expressed as the maximum probability among all possible prefetch paths of the current state Si. 0.7 is the preset probability threshold, indicating that when the probability of a certain prefetch path exceeds 70%, it is considered that this path is strongly correlated and the prefetch hit rate is relatively high.

[0106] Exemplarily, the data prefetching is converted into a scenario of reading in a library for understanding. When reading in the library, every time a page is finished, the librarian will, based on the past reading habits, guess which page might be read next and photocopy the content of those pages in advance and place it at hand. In this way, when actually turning to that page, there is no need to run to the bookshelf to find the book and one can directly read it, with extremely high speed. Here, the "current state S i " can be understood as the page being read. For example, if the current page being read is page 50 (S i = page 50), the librarian will look through the historical reading records and find that after finishing page 50 in the past: there is a 70% probability of continuing to read page 51, a 20% probability of jumping to page 100 (such as reading the ending), and a 10% probability of turning back to page 30 (such as reviewing the previous content); if a certain probability is particularly high (such as exceeding 70%), for example, the probability of page 51 here is 70%, the librarian will photocopy page 51 in advance and place it on the table for standby. If the probabilities are all very low (such as all less than 50%), perhaps no advance preparation will be made.

[0107] In some embodiments, during prefetching, prefetching can be performed according to a preset prefetch step length, and this step length can be calculated based on the average step length. For example: .

[0108] In some embodiments, during prefetching, it is also necessary to detect whether the prefetch data is valid. Specifically, CRC or data fingerprints can be verified. In the case of validity, the query data is determined. At the same time, the queue (Hot queue and Warm queue) of the prefetch data can be adjusted, and the hit rate can be recalculated.

[0109] 212. Aggregate at least one query result to obtain the target query result corresponding to the data to be queried.

[0110] In the embodiments of the present application, for the description of step 212, please refer to the detailed description of step 104 in the above embodiments, and the embodiments of the present application will not repeat it.

[0111] 213. Periodically detect the partition heat of each partition in the storage system.

[0112] In the embodiments of the present application, in each stage such as data writing, querying, and reading, access to partitions will be involved. Then, if a partition has too many access times or too much data volume, etc., it will affect the performance of the partition. Therefore, according to a certain period, the partition heat of each partition can be periodically detected.

[0113] In some embodiments, the partition heat can be expressed as:

[0114] Among them, T can represent the statistical time window, which can be a custom configuration parameter with a default value of 300s; the partition capacity can represent the maximum data capacity of a single partition, such as 1GB, which is a fixed constant predefined by the storage policy; and and are weight coefficients, + = 1, and can be automatically adjusted and optimized through the gradient descent method.

[0115] 214. When the partition heat of the third partition meets the preset migration condition, the data in the third partition is migrated to the fourth partition.

[0116] In the embodiment of the present application, the preset migration condition can be expressed as:

[0117] Among them, can represent the average heat of all partitions, obtained through periodic detection; can represent the heat standard deviation, can represent the initial adjustment coefficient, with an initial value of 1.5, which can be custom adjusted to control the migration frequency.

[0118] It should be noted that if the migration is unsuccessful, a migration warning message can be output to prompt the staff for maintenance.

[0119] During migration, it is also necessary to determine the partition to which the migration is made. The fourth partition can be determined according to the load variance value of at least one controller node. Specifically, it can be expressed as:

[0120]

[0121] Among them, can represent the heat of the k-th partition of node j currently; can represent the heat change of node j caused by migration, which can be understood as only the heat of the migrated node increases by Hi, and the heat of other nodes remains unchanged; can represent the heat value of the partition to be migrated; M can represent the number of partitions currently managed by node j, which may change after migration.

[0122] It should be noted that when the partition is migrated to node j, at this time, node j will add a partition with a heat of H i So, its total heat will increase by Hi, that is: ΔH j = H i, which means this node bears the additional load brought by the migration. When the partition is not migrated to node j, the loads of other nodes will not be affected by this migration, and their heat levels remain unchanged. Therefore: ΔH j = 0, which means these nodes are not affected by this migration and their loads remain the same.

[0123] It should be noted that in order to accurately evaluate the load distribution of the entire system after migration when making load balancing decisions and implement the core logic of the target node selection algorithm, when selecting the node to migrate to, a node j can be found: after it receives the new partition, the load variance of all nodes in the entire system is the smallest (that is, the load is the most balanced). Therefore, it is necessary to determine which node will have an additional load of H i (that is, ΔH j = H i ), and which nodes will not change (that is, ΔH j = 0). Only in this way can the new load distribution after migration be calculated and which migration method is the most balanced be compared. In addition, by setting ΔH j to H i or 0, it can be clearly expressed that partition migration is a discrete, one-to-one action that will not have a fuzzy impact on multiple nodes, making it easier to model and calculate the global load distribution after migration.

[0124] It should be noted that the purpose of migration is to optimize system performance and resource utilization. Some partitions may have too high a load (hot partitions), while other partitions have a low load. Through migration, data can be transferred from high-load partitions to low-load partitions to achieve load balancing; some partitions may over-occupy resources (such as storage space and computing power), while other partitions have idle resources. Through migration, resources can be more reasonably allocated to improve overall utilization; the query performance of some partitions may be poor (such as long response times). Through migration, data can be transferred to partitions with better performance to improve query efficiency; when a partition fails, data can be transferred to other partitions through migration to ensure the high availability of the system; the data distribution of some partitions may be uneven (such as data skew). Through migration, the data distribution can be adjusted to make the system process queries more efficiently.

[0125] It should be noted that the partition to which data can be migrated can be a partition with low load, which has a low load and can receive additional data without affecting performance; it can be a partition with high resource utilization, with sufficient resources (such as storage space and computing power) to support the storage and processing of new data; it can be a partition with high performance, with good performance (such as short response time and high throughput) to efficiently process new data; it can be a partition with low data skew, with uniform data distribution and no data skew caused by receiving new data; it can be a stable partition, with stable operation, no faults or performance fluctuations, and can support new data for a long time.

[0126] In this solution, data migration is achieved by detecting the partition heat. By reflecting the change of access frequency over time, the current heat of the partition can be evaluated more accurately, and data migration can achieve load balancing among partitions and improve data response efficiency.

[0127] In the embodiments of the present application, the core architecture of the storage system is also disclosed to implement the above solution. The core architecture may include: (1) Lock-free ASRH hybrid index layer: Based on partition-level atomic operations (CAS) and version number control, global lock contention is eliminated. This is the basic layer of the entire architecture, responsible for data storage and retrieval. It combines hash and B+ tree to achieve efficient point query and range query.

[0128] (2) Dynamic load balancing layer: Lightweight atomic migration protocol to avoid lock blocking during the migration process. This layer depends on the data partition information of the ASRH hybrid index layer. By monitoring the heat of each partition, it dynamically adjusts the data distribution to optimize system performance.

[0129] (3) Adaptive cache layer: Dual LRU queue + Markov prefetch model. This layer is closely related to the ASRH hybrid index layer and the dynamic load balancing layer. It uses the data access pattern of the index layer and the heat information of the load balancing layer to optimize the data caching strategy and improve data access efficiency.

[0130] (4) Streaming compression layer: Joint optimization of LZ4 compression and data block merging. This layer is located at the top of the architecture. It compresses the data processed by the first three layers to reduce the storage space and network transmission overhead. It depends on the data structure of the index layer and the access pattern of the cache layer to optimize the compression strategy.

[0131] In some embodiments, data interaction can be achieved among the above four-layer structures, and it can be described by changing the scenario.

[0132] The ASRH index operation process can be similar to a sorting center, which needs to quickly receive packages (data writing) and attach the correct labels (LBA addresses). When a certain shelf (partition) is full, the sorter (CAS operation) will notify the dispatching department (dynamic load balancing layer): "Shelf A is full, and expansion or transfer of goods is required!", if there is a conflict in the package labels (hash conflict), the sorter will merge adjacent packages (LBA interval merging) to avoid blocking the conveyor belt.

[0133] The dynamic load balancing process can be similar to a dispatching center, which needs to monitor which shelves are the busiest (heat score) and adjust resource allocation. After receiving the notice from the sorting center that "Shelf A is full", the dispatcher immediately expands the capacity: adds a temporary storage area to Shelf A (data migration); transfers goods: moves some packages on Shelf A to the idle Shelf B (load balancing); and at the same time notifies the courier (prefetch engine): "Next, the customer may need the packages on Shelf B. Prepare the goods in advance!".

[0134] The prefetch engine process is an intelligent courier, which needs to predict customer needs based on historical orders and pack the packages in advance (data prefetch). When the dispatching center prompts that "Shelf B may be in hot sale", the courier immediately checks the historical orders: it is found that 70% of the customers who buy the goods on Shelf B will also buy the goods on Shelf C; prepares the goods in advance: puts the packages on Shelves B and C together into the delivery vehicle (cache). If the prediction is accurate, the vehicle can be directly dispatched when the customer places an order, and the speed is increased by 3 times; if the prediction is incorrect, at most one more item will be placed in the vehicle, and the resource waste is less than 10%.

[0135] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner.

[0136] As Figure 3 shown, the embodiments of the present application further provide a data query device for a storage system. The data query device for the storage system may include: An obtaining module 301, configured to obtain an address range of data to be queried; A processing module 302, configured to determine at least one target partition corresponding to the address range according to pre-stored data partition information of the storage system, where the data partition information is obtained by pre-dividing the data of the storage system according to the logical block address range; The processing module 302 is further configured to, when querying data in a first partition among at least one target partition, determine a second partition to be queried next corresponding to the first partition according to a pre-stored transition probability matrix, and perform data prefetch on the second partition to obtain at least one query result, where the elements in the transition probability matrix are used to represent the prefetch probability between various data in the storage system; The processing module 302 is further configured to summarize at least one query result to obtain a target query result corresponding to the data to be queried.

[0137] In some embodiments, the obtaining module 301 is further configured to obtain data to be written. The processing module 302 is further configured to map the data to be written according to a preset hash function, store the data to be written into a corresponding partition, and obtain data partition information. The processing module 302 is further configured to save the data partition information.

[0138] In some embodiments, the processing module 302 is specifically configured to map the data to be written according to a preset hash function to obtain a mapping result. The processing module 302 is specifically configured to, if the mapping result indicates that there are partition conflicts for at least two conflicting data, merge the at least two conflicting data to obtain a data merging interval. The processing module 302 is specifically configured to update the version number according to the data merging interval and store the data merging interval to obtain data partition information.

[0139] In some embodiments, the processing module 302 is specifically configured to detect the data transmission delay situation in real time. The processing module 302 is specifically configured to determine a data compression level according to the data transmission delay situation. The processing module 302 is specifically configured to compress the data to be written according to the data compression level and store it into a corresponding partition to obtain data partition information.

[0140] In some embodiments, the processing module 302 is further configured to periodically detect the partition heat of each partition in the storage system. The processing module 302 is further configured to, when the partition heat of the third partition meets a preset migration condition, migrate the data in the third partition to the fourth partition, where the fourth partition is determined according to the load variance value of at least one controller node.

[0141] In some embodiments, the obtaining module 301 is further configured to obtain a historical access sequence. The processing module 302 is further configured to count the number of transfers between the first data and the second data according to the historical access sequence, where the first data and the second data are both arbitrary data in the storage system. The processing module 302 is further configured to determine a prefetch probability between the first data and the second data according to the number of transfers. The processing module 302 is further configured to summarize the prefetch probabilities to obtain a transfer probability matrix.

[0142] In some embodiments, the processing module 302 is specifically configured to, when querying data in the first partition, retrieve the prefetch probability corresponding between the first partition and at least one other partition from the transition probability matrix. The processing module 302 is specifically configured to, if the prefetch probability between the first partition and the second partition is greater than a preset probability threshold and greater than the prefetch probability between the first partition and other partitions, perform data prefetch on the second partition to obtain at least one query result.

[0143] In the embodiments of the present application, for the descriptions of the features corresponding to the data query device of the storage system, reference can be made to the relevant descriptions of the corresponding embodiments of the data query method of the storage system, which will not be elaborated here one by one.

[0144] As Figure 4 shown, the embodiments of the present application further provide an electronic device, including a memory 401 and a processor 402. A computer program is stored in the memory 401, and the processor 402 is configured to run the computer program to execute the steps in any of the above embodiments of the data query method of the storage system.

[0145] The embodiments of the present application further provide a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any of the above embodiments of the data query method of the storage system when running.

[0146] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk or optical disc and other various media that can store computer programs.

[0147] The embodiments of the present application further provide a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the data query method of the storage system.

[0148] The embodiments of the present application further provide another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the data query method of the storage system.

[0149] Those skilled in the art may further realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0150] The above has introduced in detail the process monitoring of a storage system provided by this application. Specific examples are used herein to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for querying data in a storage system, characterized in that, The method includes: Obtaining an address range of data to be queried; Determining at least one target partition corresponding to the address range according to pre-stored data partition information of a storage system, where the data partition information is obtained by previously dividing the data of the storage system according to a logical block address range; When querying data in a first partition among the at least one target partition, determining a second partition to be queried next corresponding to the first partition according to a pre-stored transition probability matrix, and pre-reading data in the second partition to obtain at least one query result, where elements in the transition probability matrix are used to represent the pre-reading probability between various data in the storage system; Summarizing the at least one query result to obtain a target query result corresponding to the data to be queried.

2. The method according to claim 1, characterized in that, Before determining at least one target partition corresponding to the address range according to the pre-stored data partition information of the storage system, the method further includes: Obtaining data to be written; Mapping the data to be written according to a preset hash function, and storing the data to be written in a corresponding partition to obtain the data partition information; Saving the data partition information.

3. The method according to claim 2, wherein The mapping the data to be written according to a preset hash function, and storing the data to be written in a corresponding partition to obtain the data partition information includes: Mapping the data to be written according to the preset hash function to obtain a mapping result; When the mapping result indicates that there are partition conflicts among at least two conflicting data, merging the at least two conflicting data to obtain a data merging range; Updating a version number according to the data merging range, and storing the data merging range to obtain the data partition information.

4. The method according to claim 2, wherein The storing the data to be written in a corresponding partition to obtain the data partition information includes: Real-time detecting a data transmission delay situation; Determining a data compression level according to the data transmission delay situation; Compressing the data to be written according to the data compression level and storing the compressed data in a corresponding partition to obtain the data partition information.

5. The method according to claim 1, wherein The method further includes: Periodically detecting partition hotness of each partition in the storage system; When the partition hotness of a third partition meets a preset migration condition, migrating data in the third partition to a fourth partition, where the fourth partition is determined according to a load variance value of at least one controller node.

6. The method according to claim 1, characterized in that, Before querying data in a first partition among the at least one target partition, determining a second partition to be queried next corresponding to the first partition according to a pre-stored transition probability matrix, and pre-reading data in the second partition to obtain at least one query result, the method further includes: Obtaining a historical access sequence; Counting the number of transitions between a first data and a second data according to the historical access sequence, where the first data and the second data are both arbitrary data in the storage system; Determining a pre-reading probability between the first data and the second data according to the number of transitions; Summarizing the pre-reading probability to obtain the transition probability matrix.

7. The method according to claim 6, wherein When performing data query on the first partition in the at least one target partition, according to the pre-stored transition probability matrix, determine the second partition to be queried next corresponding to the first partition, and perform data prefetching on the second partition to obtain at least one query result, including: When performing data query on the first partition, retrieve the prefetch probability corresponding to the first partition and at least one other partition from the transition probability matrix; If the prefetch probability between the first partition and the second partition is greater than the preset probability threshold and greater than the prefetch probability between the first partition and other partitions, perform data prefetching on the second partition to obtain the at least one query result.

8. A data query device for a storage system, characterized in that, The device includes: An acquisition module, configured to acquire the address range of the data to be queried; A processing module, configured to determine at least one target partition corresponding to the address range according to the pre-stored data partition information of the storage system, where the data partition information is obtained by pre-dividing the data of the storage system according to the logical block address range; The processing module is further configured to, when performing data query on the first partition in the at least one target partition, determine the second partition to be queried next corresponding to the first partition according to the pre-stored transition probability matrix, and perform data prefetching on the second partition to obtain at least one query result, where the elements in the transition probability matrix are used to represent the prefetch probability between each data in the storage system; The processing module is further configured to summarize the at least one query result to obtain the target query result corresponding to the data to be queried.

9. An electronic device, characterized in that, Including: A memory, configured to store a computer program; A processor, configured to implement the steps of the data query method of the storage system according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, where the computer program implements the steps of the data query method of the storage system according to any one of claims 1 to 7 when executed by a processor.

Images