A data prefetching method and apparatus, an electronic device, and a storage medium

Abstract

The application discloses a data prefetching method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a data prefetching instruction, the data prefetching instruction being used for prefetching accessed data with an access frequency greater than a preset frequency from a target disk; calling an access address sequence corresponding to the accessed data according to the data prefetching instruction; determining a sequence unit satisfying a prefetching condition as a target sequence unit according to an access condition probability, and determining a target logical block address corresponding to a target sequence identifier of the target sequence unit; obtaining a target logical block matched with the target logical block address from the target disk, and writing target data in the target logical block into a target storage medium. According to the application, the logical block satisfying the prefetching condition is obtained from the target disk according to the access condition probability of the logical block address, and the data stored in the logical block is written into the target storage medium, so that the data in the disk no longer needs to be read compared with the prior art, the access path is shortened, and the data reading efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data prefetching method, apparatus, electronic device, and storage medium. Background Technology

[0002] Distributed storage systems typically use caching to shorten the foreground I / O path, thereby reducing latency and improving system performance. For example... Figure 1 As shown, if read IO needs to read data from disk, the IO path will be very long, requiring it to pass through various modules in the interface layer, acceleration layer, and storage layer, and also requires multiple network connections. The long access path leads to increased latency. The current available solutions are to increase the physical space of the read cache, such as by using more memory or by using SCM / NVMe SSD to cache data. Increasing the read cache space can improve the random read hit rate, but the effect is minimal. Summary of the Invention

[0003] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this application provides a data prefetching method, apparatus, electronic device and storage medium.

[0004] According to one aspect of the embodiments of this application, a data prefetching method is provided, including:

[0005] Obtain a data prefetch instruction, wherein the data prefetch instruction is used to prefetch accessed data from a target disk that has been accessed more than a preset number of times, and the target disk is deployed in the storage layer of the distributed storage system;

[0006] According to the data prefetch instruction, the access address sequence corresponding to the accessed data is invoked, wherein the access address sequence includes multiple sequence units, each sequence unit includes a sequence identifier associated with the accessed data and an access condition probability, the access condition probability is calculated based on the access parameters of the logical block address, the access parameters include: the number of accesses corresponding to the logical block address;

[0007] The sequence unit whose access condition probability satisfies the prefetch condition is determined as the target sequence unit, and the target logical block address corresponding to the target sequence identifier of the target sequence unit is determined.

[0008] The target logical block that matches the target logical block address is obtained from the target disk, and the target data in the target logical block is written into the target storage medium, wherein the target storage medium is deployed in the acceleration layer of the distributed system.

[0009] Furthermore, before invoking the access address sequence corresponding to the accessed data according to the data prefetch instruction, the method further includes:

[0010] Obtain historical access records, wherein the historical access records include: multiple accessed logical block addresses, and the access time and access parameters corresponding to each logical block address;

[0011] Calculate the access condition probability corresponding to the logical block address based on the access parameters;

[0012] Based on the access parameters and access condition probabilities corresponding to each logical block address, a sequence unit corresponding to the logical block address is generated;

[0013] The sequence units corresponding to the logical block address are sorted according to the sequence unit identifier to generate the access address sequence.

[0014] Furthermore, the access parameters include: calculating the access condition probability corresponding to the logical block address based on the access parameters, including:

[0015] The total number of accesses is obtained by summing the access counts corresponding to all logical block addresses in the historical access records.

[0016] Based on the number of accesses corresponding to the logical block address and the total number of accesses, calculate the first access probability corresponding to the logical block address;

[0017] Obtain the second access probability corresponding to the logical block address, wherein the second access probability is the probability that the logical block address is accessed under random conditions;

[0018] The access condition probability is calculated based on the first access probability and the second access probability.

[0019] Furthermore, the step of generating a sequence unit based on each logical block address, the access parameters, and the access condition probability includes:

[0020] The logical block address is divided according to a preset size to obtain the address code;

[0021] The sequence unit identifier is obtained by calculating based on the address code;

[0022] The sequence unit is generated based on the sequence unit identifier, the access parameters, and the access condition probability.

[0023] Furthermore, determining the sequence unit whose access condition probability satisfies the prefetch condition as the target sequence unit includes:

[0024] Sequence units whose access condition probability is greater than or equal to a preset probability are identified as candidate sequence units;

[0025] According to the candidate access condition probability corresponding to the candidate sequence unit;

[0026] The candidate sequence units are sorted according to the candidate access conditional probability to obtain the target sequence unit.

[0027] Furthermore, after writing the target data in the target logical block into the target storage medium, the method further includes:

[0028] Receive a data read request, wherein the data read request carries read conditions;

[0029] According to the data read request, read the target data that meets the read conditions from the target storage medium;

[0030] The target data that meets the reading conditions is sent to the requester corresponding to the data reading request.

[0031] Furthermore, after writing the target data in the target logical block into the target storage medium, the method further includes:

[0032] Detect the access frequency of each target data in the target storage medium;

[0033] Delete the target data whose access frequency is less than the preset access frequency.

[0034] According to another aspect of the embodiments of this application, a data prefetching apparatus is also provided, comprising:

[0035] The acquisition module is used to acquire data prefetch instructions, wherein the data prefetch instructions are used to prefetch accessed data from a target disk that has been accessed more than a preset number of times, and the target disk is deployed in the storage layer of a distributed storage system;

[0036] The response module is used to call the access address sequence corresponding to the accessed data according to the data prefetch instruction. The access address sequence includes multiple sequence units, each sequence unit including a sequence identifier associated with the accessed data and an access condition probability. The access condition probability is calculated based on the access parameters of the logical block address. The access parameters include the number of accesses corresponding to the logical block address.

[0037] The determination module is used to determine the sequence unit whose access condition probability satisfies the prefetch condition as the target sequence unit, and to determine the target logical block address corresponding to the target sequence identifier of the target sequence unit;

[0038] The processing module is used to obtain a target logical block that matches the address of the target logical block from the target disk, and write the target data in the target logical block into the target storage medium, wherein the target storage medium is deployed in the acceleration layer of the distributed system.

[0039] In this embodiment of the application, the data prefetching device further includes: a generation module, used to acquire historical access records, wherein the historical access records include: multiple accessed logical block addresses, and the access time and access parameters corresponding to each logical block address; calculate the access condition probability corresponding to the logical block address based on the access parameters; generate a sequence unit corresponding to the logical block address according to the access parameters and access condition probability corresponding to each logical block address; sort the sequence units corresponding to the logical block address according to the sequence unit identifier to generate an access address sequence.

[0040] In this embodiment of the application, the access parameters include: the number of accesses corresponding to the logical block address;

[0041] The response module is used to sum the access counts corresponding to all logical block addresses in the historical access records to obtain the total number of accesses; calculate the first access probability corresponding to the logical block address based on the access counts corresponding to the logical block address and the total number of accesses; obtain the second access probability corresponding to the logical block address, wherein the second access probability is the probability that the logical block address is accessed under random conditions; and calculate the access condition probability based on the first access probability and the second access probability.

[0042] In this embodiment, the response module is used to divide the logical block address according to a preset size to obtain the address code; calculate based on the address code to obtain the sequence unit identifier; and generate the sequence unit based on the sequence unit identifier, access parameters, and access condition probability.

[0043] In this embodiment of the application, the determining module is used to determine the sequence units whose access condition probability is greater than or equal to a preset probability as candidate sequence units; according to the candidate access condition probability corresponding to the candidate sequence units; and sort the candidate sequence units according to the candidate access condition probability to obtain the target sequence unit.

[0044] In this embodiment of the application, the data prefetching device further includes: a processing module, configured to receive a data read request, wherein the data read request carries read conditions; read target data that meets the read conditions from the target storage medium according to the data read request; and send the target data that meets the read conditions to the requester corresponding to the data read request.

[0045] In this embodiment of the application, the data prefetching device further includes: a deletion module, used to detect the access frequency corresponding to each target data in the target storage medium; and delete the target data whose access frequency is less than a preset access frequency.

[0046] According to another aspect of the embodiments of this application, a storage medium is also provided, the storage medium including a stored program that executes the above steps when the program is run.

[0047] According to another aspect of the embodiments of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the programs stored in the memory.

[0048] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the above-described method.

[0049] Compared with the prior art, the above-mentioned technical solution provided in this application has the following advantages: The method provided in this application obtains logical blocks that meet the prefetch conditions from the target disk according to the access condition probability of the logical block address, and writes the data stored in the logical block into the target storage medium. It no longer needs to read data from the disk, shortens the access path, reduces data reading latency, and improves data reading efficiency. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A schematic diagram of a distributed storage system provided in an embodiment of this application;

[0053] Figure 2 A flowchart illustrating a data prefetching method provided in an embodiment of this application;

[0054] Figure 3 A schematic diagram illustrating the access order corresponding to logical block addresses provided in embodiments of this application;

[0055] Figure 4 A schematic diagram of the access address sequence provided in an embodiment of this application;

[0056] Figure 5A flowchart illustrating a data prefetching method provided in another embodiment of this application;

[0057] Figure 6 A flowchart illustrating a data prefetching method provided in another embodiment of this application;

[0058] Figure 7 A block diagram of a data prefetching device provided in an embodiment of this application;

[0059] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another similar entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0062] This application provides a data prefetching method, apparatus, electronic device, and storage medium. The method provided by this invention can be applied to any electronic device as needed, such as a server, terminal, or other electronic device. No specific limitation is made here, and for ease of description, it will be referred to as an electronic device below.

[0063] According to one aspect of the embodiments of this application, a method embodiment of a data prefetching method is provided. Figure 2 A flowchart of a data prefetching method provided in an embodiment of this application is shown below. Figure 2 As shown, the method includes:

[0064] Step S11: Obtain a data prefetch instruction, wherein the data prefetch instruction is used to prefetch accessed data from the target disk that has been accessed more than a preset number of times, and the target disk is deployed in the storage layer of the distributed storage system.

[0065] The method provided in this application embodiment is applied to a distributed storage system, such as... Figure 2 As shown, a distributed storage system includes an interface layer, an acceleration layer, and a storage layer. In existing technologies, when a requester needs to read data from a disk deployed in the storage layer, the data read request is typically sent to the interface layer, which then forwards it to the disk in the storage layer via the acceleration layer, thus achieving data read. This approach results in a long access path and low data read efficiency.

[0066] Based on this, this application embodiment deploys a timer within the distributed storage system. The timer periodically triggers data prefetching instructions to pre-extract data from the target disk that has been accessed more than a preset number of times. Subsequently, the pre-extracted data can be filtered, and the filtered data can be stored in the target storage medium (Read Cache) within the acceleration layer. This shortens the access path and improves data access efficiency. The target storage medium can be memory or other high-performance media.

[0067] Step S12: According to the data prefetch instruction, call the access address sequence corresponding to the accessed data. The access address sequence includes multiple sequence units, and each sequence unit includes a sequence identifier associated with the accessed data and an access condition probability.

[0068] In this embodiment of the application, the process of generating the access address sequence includes the following steps A1-A4:

[0069] Step A1: Obtain historical access records, which include: multiple accessed logical block addresses, and the access time and access parameters corresponding to each logical block address.

[0070] In the application embodiment, when the requester reads data from the disk, the distributed storage system records information such as the time the data was accessed and the logical block address of the logical block where the data is located. Then, historical access records are generated based on the above information.

[0071] Step A2: Calculate the access condition probability corresponding to the logical block address based on the access parameters.

[0072] In this embodiment of the application, the access parameters include: the number of accesses corresponding to the logical block address;

[0073] In this embodiment of the application, step A2, calculating the access condition probability corresponding to the logical block address based on the access parameters, includes the following steps A201-A204:

[0074] Step A201: Sum the access counts corresponding to all logical block addresses in the historical access records to obtain the total number of accesses.

[0075] Step A202: Calculate the first access probability corresponding to the logical block address based on the number of accesses corresponding to the logical block address and the total number of accesses.

[0076] Step A203: Obtain the second access probability of the logical block address being randomly accessed.

[0077] Step A204: Calculate the access conditional probability based on the first access probability and the second access probability.

[0078] As an example, such as Figure 3 As shown, the access order of logical block addresses can be represented as a sequence, where letters from A to Z represent logical block addresses, and the arrow direction indicates the time order of access. Then, based on this sequence, the access count for each logical block address is counted, and the total number of accesses is determined. Next, based on the access count for each logical block address and the total number of accesses, the first access probability for each logical block address is calculated. Then, the second access probability for each logical block address being randomly accessed is obtained. Finally, the conditional probability of each logical block address being randomly accessed is calculated using a Bayesian algorithm.

[0079] Step A3: Generate the sequence unit corresponding to the logical block address based on the access parameters and access condition probabilities corresponding to each logical block address.

[0080] In this embodiment of the application, step A3, generating sequence units based on each logical block address, access parameters, and access condition probability, includes the following steps A301-A303:

[0081] Step A301: Divide the logical block address according to the preset size to obtain the address code.

[0082] Step A302: Calculate the sequence unit identifier based on the address code.

[0083] Step A303: Generate sequence units based on sequence unit identifiers, access parameters, and access condition probabilities.

[0084] In this embodiment, the logical block address is divided into 64K segments to obtain the address code. The calculation process is: m = LBA / 64, where m is the address code and LBA is the logical block address. Then, the identifier of the read sequence unit (i.e., the ID of the sequence unit) is calculated based on the address code. The calculation process is: ID = m / 1024. Then, the sequence unit is generated based on the sequence unit identifier, access parameters, and access condition probability.

[0085] Step A4: Sort the sequence units corresponding to the logical block address according to the sequence unit identifier to generate the access address sequence.

[0086] In this embodiment of the application, after determining the sequence unit identifier, the sequence units are sorted according to the sequence unit identifier to obtain the final access address sequence, such as... Figure 4 As shown, each access unit sequence includes: sequence unit identifier, access count, access frequency, and access condition probability.

[0087] Step S13: Determine the sequence unit whose access condition probability satisfies the prefetch condition as the target sequence unit, and determine the target logical block address corresponding to the target sequence identifier of the target sequence unit.

[0088] In this embodiment of the application, step S13, determining the sequence unit whose access condition probability satisfies the prefetch condition as the target sequence unit, includes the following steps B1-B3:

[0089] Step B1: Sequence units whose access conditional probability is greater than or equal to a preset probability are identified as candidate sequence units.

[0090] Step B2, according to the candidate access condition probability corresponding to the candidate sequence unit.

[0091] Step B3: Sort the candidate sequence units according to the candidate access conditional probability to obtain the target sequence unit.

[0092] In this embodiment of the application, in order to improve access efficiency, sequence units with access condition probabilities greater than or equal to preset probabilities are determined as candidate sequence units. These candidate sequence units are then sorted according to their access condition probabilities, for example, such as... Figure 4 As shown, a preset number of candidate sequence units are selected as target sequence units in descending order of access condition probability.

[0093] Step S14: Obtain the target logical block that matches the target logical block address from the target disk, and write the target data in the target logical block into the target storage medium, wherein the target storage medium is deployed in the acceleration layer of the distributed system.

[0094] In this embodiment of the application, after determining the target sequence unit, since the target sequence identifier of the target sequence unit is calculated based on the logical block address, the target logical block address can be determined based on the target sequence identifier. Then, the target logical block is searched from the target disk according to the target logical block address, and the data stored in the target logical block is written to the target storage medium.

[0095] The method provided in this application embodiment retrieves logical blocks that meet the prefetch conditions from the target disk based on the access condition probability of the logical block address, and writes the data stored in the logical block into the target storage medium. This eliminates the need to read data from the disk, shortens the access path, reduces data reading latency, and improves data reading efficiency.

[0096] In the embodiments of this application, Figure 5 A flowchart of a data prefetching method provided in an embodiment of this application is shown below. Figure 5 As shown, after writing the target data in the target logical block to the target storage medium, the method further includes:

[0097] Step S21: Receive a data read request, wherein the data read request carries the read conditions.

[0098] Step S22: Based on the data read request, read the target data that meets the read conditions from the target storage medium.

[0099] Step S23: Send the target data that meets the reading conditions to the requester corresponding to the data reading request.

[0100] In this embodiment of the application, after receiving a data read request, the distributed storage system first checks whether the target storage medium in the acceleration layer has completed writing the target data. If the writing is completed, the distributed storage medium will respond to the data read request and read the target data that meets the read conditions from the target storage medium. If there is target data that meets the read conditions in the target storage medium, the target data that meets the read conditions will be sent to the requester corresponding to the data read request.

[0101] In this embodiment of the application, if there is no target data that meets the reading conditions in the target storage medium, the distributed storage system reads the data that meets the reading conditions from the disk in the storage layer and feeds the data back to the requester corresponding to the data reading request.

[0102] In the embodiments of this application, Figure 6 A flowchart of a data prefetching method provided in an embodiment of this application is shown below. Figure 6 As shown, after writing the target data in the target logical block to the target storage medium, the method further includes:

[0103] Step S31: Detect the access frequency corresponding to each target data in the target storage medium.

[0104] Step S32: Delete target data whose access frequency is less than the preset access frequency.

[0105] In this embodiment of the application, the distributed storage system periodically detects the access frequency of each target data in the target storage medium, or when the remaining storage space of the target storage medium is less than the preset storage space, the distributed storage system detects the access frequency of each target data in the target storage medium.

[0106] In this embodiment of the application, by deleting target data whose access frequency is less than a preset frequency, the target data with low utilization in the target storage medium can be cleared in a timely manner, thereby freeing up storage space and providing storage space for subsequent data prefetching from the disk.

[0107] Figure 7 This is a block diagram of a data prefetching device provided in an embodiment of this application. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 7 As shown, the device includes:

[0108] The acquisition module 41 is used to acquire data prefetch instructions, wherein the data prefetch instructions are used to prefetch accessed data from the target disk that has been accessed more than a preset number of times, and the target disk is deployed in the storage layer of the distributed storage system;

[0109] The response module 42 is used to call the access address sequence corresponding to the accessed data according to the data prefetch instruction. The access address sequence includes multiple sequence units, and each sequence unit includes a sequence identifier associated with the accessed data and an access condition probability.

[0110] The determination module 43 is used to determine the sequence unit whose access condition probability satisfies the prefetch condition as the target sequence unit, and to determine the target logical block address corresponding to the target sequence identifier of the target sequence unit;

[0111] Processing module 44 is used to obtain a target logical block that matches the address of the target logical block from the target disk, and write the target data in the target logical block into the target storage medium, wherein the target storage medium is deployed in the acceleration layer of the distributed system.

[0112] In this embodiment of the application, the data prefetching device further includes: a generation module, used to acquire historical access records, wherein the historical access records include: multiple accessed logical block addresses, and the access time and access parameters corresponding to each logical block address; calculate the access condition probability corresponding to the logical block address based on the access parameters; generate a sequence unit corresponding to the logical block address according to the access parameters and access condition probability corresponding to each logical block address; sort the sequence units corresponding to the logical block address according to the sequence unit identifier to generate an access address sequence.

[0113] In this embodiment of the application, the access parameters include: the number of accesses corresponding to the logical block address;

[0114] The response module 42 is used to sum the access counts corresponding to all logical block addresses in the historical access records to obtain the total number of accesses; calculate the first access probability corresponding to the logical block address based on the access counts corresponding to the logical block address and the total number of accesses; obtain the second access probability corresponding to the logical block address, wherein the second access probability is the probability that the logical block address is accessed under random conditions; and calculate the access condition probability based on the first access probability and the second access probability.

[0115] In this embodiment of the application, the response module 42 is used to divide the logical block address according to a preset size to obtain the address code; calculate based on the address code to obtain the sequence unit identifier; and generate the sequence unit based on the sequence unit identifier, access parameters, and access condition probability.

[0116] In this embodiment of the application, the determining module 43 is used to determine the sequence units whose access condition probability is greater than or equal to a preset probability as candidate sequence units; according to the candidate access condition probability corresponding to the candidate sequence units; and sort the candidate sequence units according to the candidate access condition probability to obtain the target sequence unit.

[0117] In this embodiment of the application, the data prefetching device further includes: a processing module, configured to receive a data read request, wherein the data read request carries read conditions; read target data that meets the read conditions from the target storage medium according to the data read request; and send the target data that meets the read conditions to the requester corresponding to the data read request.

[0118] In this embodiment of the application, the data prefetching device further includes: a deletion module, used to detect the access frequency corresponding to each target data in the target storage medium; and delete the target data whose access frequency is less than a preset access frequency.

[0119] This application also provides an electronic device, such as... Figure 8As shown, the electronic device may include: a processor 1501, a communication interface 1502, a memory 1503, and a communication bus 1504, wherein the processor 1501, the communication interface 1502, and the memory 1503 communicate with each other through the communication bus 1504.

[0120] Memory 1503 is used to store computer programs;

[0121] When the processor 1501 executes the computer program stored in the memory 1503, it implements the steps of the above embodiments.

[0122] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0123] The communication interface is used for communication between the aforementioned terminal and other devices.

[0124] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0125] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0126] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the data prefetching methods described in the above embodiments.

[0127] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the data prefetching methods described in the above embodiments.

[0128] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk).

[0129] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

[0130] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data prefetching method, characterized in that, Applied to a distributed storage system, the method includes: Obtain a data prefetch instruction, wherein the data prefetch instruction is used to prefetch accessed data from a target disk that has been accessed more than a preset number of times, and the target disk is deployed in the storage layer of the distributed storage system; According to the data prefetch instruction, the access address sequence corresponding to the accessed data is invoked, wherein the access address sequence includes multiple sequence units, each sequence unit includes a sequence identifier associated with the accessed data and an access condition probability, the access condition probability is calculated based on the access parameters of the logical block address, the access parameters include: the number of accesses corresponding to the logical block address; The sequence unit whose access condition probability satisfies the prefetch condition is determined as the target sequence unit, and the target logical block address corresponding to the target sequence identifier of the target sequence unit is determined. The target logical block that matches the target logical block address is obtained from the target disk, and the target data in the target logical block is written into the target storage medium, wherein the target storage medium is deployed in the acceleration layer of the distributed storage system.

2. The method according to claim 1, characterized in that, Before invoking the access address sequence corresponding to the accessed data according to the data prefetch instruction, the method further includes: Obtain historical access records, wherein the historical access records include: multiple accessed logical block addresses, and the access time and access parameters corresponding to each logical block address; Calculate the access condition probability corresponding to the logical block address based on the access parameters; Based on the access parameters and access condition probabilities corresponding to each logical block address, a sequence unit corresponding to the logical block address is generated; The sequence units corresponding to the logical block address are sorted according to the sequence unit identifier to generate the access address sequence.

3. The method according to claim 2, characterized in that, The step of calculating the access condition probability corresponding to the logical block address based on the access parameters includes: The total number of accesses is obtained by summing the access counts corresponding to all logical block addresses in the historical access records. Based on the number of accesses corresponding to the logical block address and the total number of accesses, calculate the first access probability corresponding to the logical block address; Obtain the second access probability corresponding to the logical block address, wherein the second access probability is the probability that the logical block address is accessed under random conditions; The access condition probability is calculated based on the first access probability and the second access probability.

4. The method according to claim 2, characterized in that, The step of generating a sequence unit based on each logical block address, the access parameters, and the access condition probability includes: The logical block address is divided according to a preset size to obtain the address code; The sequence unit identifier is obtained by calculating based on the address code; The sequence unit is generated based on the sequence unit identifier, the access parameters, and the access condition probability.

5. The method according to claim 1, characterized in that, The step of determining the sequence unit whose access condition probability satisfies the prefetch condition as the target sequence unit includes: Sequence units whose access condition probability is greater than or equal to a preset probability are identified as candidate sequence units; According to the candidate access condition probability corresponding to the candidate sequence unit; The candidate sequence units are sorted according to the candidate access conditional probability to obtain the target sequence unit.

6. The method according to claim 1, characterized in that, After writing the target data in the target logical block to the target storage medium, the method further includes: Receive a data read request, wherein the data read request carries read conditions; According to the data read request, read the target data that meets the read conditions from the target storage medium; The target data that meets the reading conditions is sent to the requester corresponding to the data reading request.

7. The method according to claim 1, characterized in that, After writing the target data in the target logical block to the target storage medium, the method further includes: Detect the access frequency of each target data in the target storage medium; Delete the target data whose access frequency is less than the preset access frequency.

8. A data prefetching device, characterized in that, include: The acquisition module is used to acquire data prefetch instructions, wherein the data prefetch instructions are used to prefetch accessed data from a target disk that has been accessed more than a preset number of times, and the target disk is deployed in the storage layer of a distributed storage system; The response module is used to call the access address sequence corresponding to the accessed data according to the data prefetch instruction. The access address sequence includes multiple sequence units, each sequence unit including a sequence identifier associated with the accessed data and an access condition probability. The access condition probability is calculated based on the access parameters of the logical block address. The access parameters include the number of accesses corresponding to the logical block address. The determination module is used to determine the sequence unit whose access condition probability satisfies the prefetch condition as the target sequence unit, and to determine the target logical block address corresponding to the target sequence identifier of the target sequence unit; The processing module is used to obtain a target logical block that matches the address of the target logical block from the target disk, and write the target data in the target logical block into the target storage medium, wherein the target storage medium is deployed in the acceleration layer of the distributed storage system.

9. A storage medium, characterized in that, The storage medium includes a stored program, wherein the program, when executed, performs the method steps of any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor for executing the method steps of any one of claims 1-7 by running a program stored in memory.

Images