A skip-reading pre-reading method, device, equipment and storage medium

Abstract

The present invention discloses a skip read pre-reading method, device, equipment and storage medium, the method comprising: receiving a read request, if the data corresponding to the read request is already stored locally, then reading the data corresponding to the read request locally, otherwise, reading the data corresponding to the read request from a target file; after reading the data corresponding to the read request, determining whether the current read mode is a skip read mode, if so, predicting the pre-read data position and pre-read data length of the data to be read after the current moment based on the skip read mode, pre-reading the data corresponding to the pre-read data position and the pre-read data length from the target file, and storing the pre-read data locally, otherwise, determining that no pre-reading operation is required; the pre-reading operation is an operation of pre-reading corresponding data from the target file. Therefore, in the scenario where the read operation is implemented in the skip read mode, the present application can improve the data reading speed and the reading performance to a certain extent by pre-reading data.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed file systems, and more specifically, to a skip read and pre-read method, device, equipment and storage medium. Background Art

[0002] For distributed file systems (object storage), files are generally divided into blocks of a certain size, and index information is stored in fixed locations in different blocks. When accessing these files, the storage server of the distributed file system generally accesses the file to read the required data after receiving the read request, and then returns the required data to the sender of the read request. However, the inventors have found that since this method requires accessing the file each time to obtain the corresponding data, the rate of obtaining the corresponding data is slow, which in turn leads to poor read performance. Summary of the invention

[0003] The purpose of the present invention is to provide a skip read and pre-read method, device, equipment and storage medium, which can improve the data reading speed to a certain extent by pre-reading data in a scenario where a read operation is implemented in a skip read mode, thereby improving the reading performance.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A skip-reading pre-reading method, comprising:

[0006] Receive a read request, and if the data corresponding to the read request is already stored locally, read the data corresponding to the read request locally; otherwise, read the data corresponding to the read request from the target file;

[0007] After reading the data corresponding to the read request, determine whether the current read mode is a skip read mode. If so, predict the pre-read data position and pre-read data length of the data to be read after the current moment based on the skip read mode, pre-read the data corresponding to the pre-read data position and the pre-read data length from the target file, and store the pre-read data locally. Otherwise, determine that no pre-read operation is required; wherein the pre-read operation is an operation of pre-reading corresponding data from the target file.

[0008] Preferably, determining whether the current reading mode is a skip reading mode includes:

[0009] If the data length and step length of the data read in this read operation are the same as those of the previous read operation, then the current read mode is determined to be the skip read mode; wherein the step length of any read operation is the data length between the read start position of any read operation and the read end position of the previous read operation of any read operation, and the read operation is an operation of reading the data corresponding to any read request after receiving the any read request.

[0010] Preferably, predicting the pre-read data position and pre-read data length of data to be read after the current moment based on the skip mode includes:

[0011] If the first pre-read operation is required, the pre-read data position is determined to be the position of the read end position of the last read operation offset by the step length, and the pre-read data length is the data length of the last read operation;

[0012] If the pre-read operation currently required is the nth one of multiple consecutive pre-read operations, the pre-read data positions are determined to be positions after the read end position of the last pre-read operation is offset by 1 to n times the step length, and the pre-read data length corresponding to each of the pre-read data positions is the data length of the first pre-read operation among the multiple consecutive pre-read operations.

[0013] Preferably, after predicting the pre-read data position of the data to be read after the current moment based on the skip mode, the method further includes:

[0014] If the object used to implement the target file storage is larger than the cycle of the skip mode, each object where the pre-read data position is located is pre-read in a whole-object reading manner, and the pre-read data is stored locally; otherwise, the step of pre-reading the data corresponding to the pre-read data position and the pre-read data length in the target file is executed.

[0015] Preferably, it also includes:

[0016] After each pre-reading operation and each reading operation, the skip reading record information recorded in the skip reading record structure is updated, and the skip reading record information includes a step length, a data length, and a reading end position.

[0017] Preferably, before updating the skip record information recorded in the skip record structure, the method further includes:

[0018] Open the target file, create a handle, and create the skip record structure in the handle;

[0019] The method further comprises:

[0020] The target file and the handle are closed, and the skip record structure is cleared.

[0021] Preferably, after the data corresponding to the read request is read locally, the method further includes:

[0022] The locally stored data corresponding to the read request is cleared.

[0023] A skip-reading and pre-reading device, comprising:

[0024] The reading module is used to: receive a read request, and if the data corresponding to the read request is already stored locally, read the data corresponding to the read request locally; otherwise, read the data corresponding to the read request from a target file;

[0025] The first pre-reading module is used to: after reading the data corresponding to the read request, determine whether the current read mode is a skip read mode, and if so, predict the pre-read data position and pre-read data length of the data to be read after the current moment based on the skip read mode, pre-read the data corresponding to the pre-read data position and the pre-read data length from the target file, and store the pre-read data locally; otherwise, determine that no pre-read operation is required; wherein the pre-read operation is an operation of pre-reading corresponding data from the target file.

[0026] A skip-reading and pre-reading device, comprising:

[0027] Memory for storing computer programs;

[0028] A processor is used to implement the steps of the skip read pre-reading method as described in any one of the above items when executing the computer program.

[0029] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the skip-reading pre-reading method as described in any one of the above items are implemented.

[0030] The present invention provides a skip read pre-reading method, device, equipment and storage medium, the method comprising: receiving a read request, if the data corresponding to the read request has been stored locally, then reading the data corresponding to the read request locally, otherwise, reading the data corresponding to the read request from a target file; after reading the data corresponding to the read request, judging whether the current read mode is a skip read mode, if so, predicting the pre-read data position and pre-read data length of data to be read after the current moment based on the skip read mode, pre-reading the data corresponding to the pre-read data position and the pre-read data length from the target file, and storing the pre-read data locally, otherwise, determining that no pre-reading operation is required; wherein the pre-reading operation is an operation of pre-reading corresponding data from the target file. After reading the data corresponding to the read request each time, the present application determines whether the current read mode is the skip read mode, and then when the judgment result is yes, the corresponding data is read from the target file based on the skip read mode and stored locally, so that when the corresponding read request for this part of the data is received subsequently, the corresponding data can be read locally. Compared with accessing the file to read the data, the data reading from the local can greatly improve the data reading speed. Therefore, in the scenario where the read operation is implemented in the skip read mode, the present application can improve the data reading speed to a certain extent by pre-reading the data, thereby improving the reading performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0032] Figure 1 A flowchart of a skip read pre-reading method provided by an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of the overall flow chart of a skip-reading pre-reading method in a specific application scenario provided by an embodiment of the present invention;

[0034] Figure 3 An example diagram of identifying a skip reading mode in a skip reading and pre-reading method provided by an embodiment of the present invention;

[0035] Figure 4 An example diagram of initiating pre-reading in a skip-read pre-reading method provided by an embodiment of the present invention;

[0036] Figure 5 The present invention provides a flowchart of a skip read pre-reading method. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] See also Figure 1 , which shows a flow chart of a skip read pre-reading method provided by an embodiment of the present invention, which may include:

[0039] S11: receiving a read request. If data corresponding to the read request is already stored locally, the data corresponding to the read request is read locally. Otherwise, the data corresponding to the read request is read from the target file.

[0040] The skip-reading and pre-reading method provided in the embodiment of the present invention can be applied to a distributed file system, and the target file is any video, streaming media and other files that need to be accessed in the distributed file system; the execution subject of the skip-reading and pre-reading method can be a corresponding skip-reading and pre-reading device, and the skip-reading and pre-reading device can be set in a storage server of the distributed file system, so the execution subject of the skip-reading and pre-reading method can be a storage server, and the embodiment of the present application is specifically described with the execution subject of the skip-reading and pre-reading method being a storage server.

[0041] The storage server receives a read request. If the data corresponding to the read request is already stored locally, the storage server can read the data corresponding to the read request locally and return the data corresponding to the read request to the sender of the read request. Otherwise, the data corresponding to the read request is read from the target file and returned to the sender of the read request. The read request may include an identifier of the data to be read, so that the corresponding data can be located based on the identifier, thereby realizing data reading; in addition, any read request received by the storage server can be processed accordingly according to the scheme disclosed in this application.

[0042] S12: After reading the data corresponding to the read request, determine whether the current read mode is a skip read mode, if yes, execute step S13, if no, execute step S14.

[0043] The read mode is a mode for reading data. When judging whether the current mode is the skip read mode, it can be implemented based on the read operation characteristics of the skip read mode, or it can be implemented based on a pre-set identifier, or it can be implemented in other ways set according to actual needs. Specifically, when implemented based on the read operation characteristics of the skip read mode, since the read operation of the skip read mode generally has the same length of data read each time, and the step length when reading data twice is also the same, it can be judged based on the current read operation and the previous read operation whether the reading of the corresponding data is currently implemented in the skip read mode, wherein the step length of any read operation is the data length between the starting position of the data read by the any read operation in the target file and the ending position of the data read by the previous read operation of the any read operation in the target file; when implemented based on a pre-set identifier, it can be pre-specified whether the read operation needs to be implemented in the skip read mode at present, and a corresponding mark can be set, so that it can be known based on the mark whether the read operation needs to be implemented in the skip read mode at present.

[0044] S13: Predicting the pre-read data position and pre-read data length of data to be read after the current time based on the skip mode, pre-reading data corresponding to the pre-read data position and the pre-read data length from the target file, and storing the pre-read data locally.

[0045] If it is determined that the current mode is skip reading, then since the data length and step length of each data read in the skip reading mode are the same, the position and length of the data to be read next time or even n times thereafter can be determined based on this, and then the corresponding data can be pre-read and stored locally based on this, so that after receiving the read request corresponding to this data, the data corresponding to the read request can be directly read locally and returned, which can greatly improve the data reading speed compared to reading data from the target file.

[0046] S14: Determine that no pre-reading operation is required; wherein the pre-reading operation is an operation of pre-reading corresponding data from the target file.

[0047] In the embodiment of the present application, reading and pre-reading refer to different actions. Reading is to read the data corresponding to the read request from the local or target file, while pre-reading is to pre-read the data to be read after the current moment from the target file; correspondingly, the read operation is the operation of reading the corresponding data, that is, the operation of reading the data corresponding to the read request from the local or target file, while the pre-reading operation is the operation of pre-reading the corresponding data, that is, the operation of pre-reading the data to be read after the current moment from the target file.

[0048] After reading the data corresponding to the read request each time, the present application determines whether the current read mode is the skip read mode, and then when the judgment result is yes, the corresponding data is read from the target file based on the skip read mode and stored locally, so that when the corresponding read request for this part of the data is received subsequently, the corresponding data can be read locally. Compared with accessing the file to read the data, the data reading from the local can greatly improve the data reading speed. Therefore, in the scenario where the read operation is implemented in the skip read mode, the present application can improve the data reading speed to a certain extent by pre-reading the data, thereby improving the reading performance.

[0049] An embodiment of the present invention provides a skip read pre-reading method, which determines whether a current read mode is a skip read mode, and may include:

[0050] If the data length and step length of the data read in this read operation are the same as those of the previous read operation, then the current read mode is determined to be the skip read mode; wherein the step length of any read operation is the data length between the read start position of any read operation and the read end position of the previous read operation of any read operation, and the read operation is an operation of reading the data corresponding to any read request after receiving the any read request.

[0051] When determining whether the read operation performed at the current moment and within a period of time before the current moment is a read operation in a skip read mode, it is possible to determine whether the current read mode is a skip read mode based on whether the data length and step length of the current read operation and the previous read operation are the same, thereby determining whether it is a skip read mode through the characteristics of the two read operations, which is simple and effective. Among them, the read start position is the starting position of the read data or the pre-read data in the target file, and the read end position is the end position of the read data or the pre-read data in the target file.

[0052] An embodiment of the present invention provides a skip read pre-reading method, which predicts the pre-read data position and pre-read data length of data to be read after the current time based on a skip read mode, including:

[0053] If the first pre-read operation is required, the pre-read data position is determined to be the position after the read end position of the last read operation is offset by the step length, and the pre-read data length is the data length of the last read operation;

[0054] If the pre-read operation currently required is the nth one of multiple consecutive pre-read operations, the pre-read data positions are determined to be positions offset by 1 to n times the read end position of the last pre-read operation, and the pre-read data length corresponding to each pre-read data position is the data length of the first pre-read operation among the multiple consecutive pre-read operations.

[0055] If it is the first time that the data is determined to be in the skip reading mode, in order to avoid invalid data read due to changes in the read operation later (that is, the pre-read data is not the data that the sender of the read request needs to read), the present application can pre-read data within a skip reading mode cycle; if it is determined that the data is currently in the skip reading mode for multiple consecutive times, it means that the skip reading mode has lasted for a long time, and the possibility of changes in the read operation later is also relatively small. Therefore, the present application can pre-read data within multiple skip reading mode cycles; thus, in the above manner, when the possibility of being in the skip reading mode later is greater, more data is pre-read for reading, and a high hit rate of the pre-read data is guaranteed to a certain extent when pre-reading is implemented.

[0056] Among them, one cycle of the skip reading mode is the sum of the data length and the step length read in a certain read operation, the pre-read data position is the starting position of the data to be pre-read in the target file, and the pre-read data length is the length of the data to be pre-read in the target file; and n is a positive integer not equal to 1, so that when it is determined that the later period is still in the skip reading mode, the data within multiple cycles that may be read in the future are read, that is, the pre-read data positions are respectively the position after one step length is offset backward (backward means in the direction from the starting position to the end position of the data) from the read end position of the last pre-read data, the position after two steps lengths are offset backward, and so on, until the position after n steps lengths are offset backward, and data of the corresponding pre-read data length is read at each pre-read data position, thereby realizing effective pre-reading of data.

[0057] In addition, the first pre-reading operation can be the first pre-reading operation when no pre-reading operation has been performed before the current moment, or it can be the first pre-reading operation when there is a read operation between the last pre-reading operation and the current pre-reading operation when the pre-reading operations are not performed continuously, and the current pre-reading operation is the first pre-reading operation. Of course, other definition methods of the first pre-reading operation that meet the requirements can also be set according to actual needs, all of which are within the protection scope of the present invention.

[0058] A skip read pre-reading method provided by an embodiment of the present invention may further include:

[0059] If the object used to implement the target file storage is larger than the cycle of the skip mode, the object where each pre-read data position is located is pre-read in a whole object reading manner, and the pre-read data is stored locally; otherwise, the step of pre-reading the data corresponding to the pre-read data position and the pre-read data length in the target file is executed.

[0060] The distributed file system can be implemented in the form of object storage, and the target file is stored in the form of an object. If the size of the object is larger than the period of the skip mode, it means that the object needs to be accessed multiple times. Therefore, in order to reduce the number of object accesses, the object where the data to be pre-read is located can be read as a whole object in the embodiment of the present application, that is, the entire object where the data to be pre-read is located can be read; and when the size of the object is not larger than the period of the skip mode, the data can be pre-read in the manner of the above embodiment. In this way, for skip-intensive requests (multiple small-block skips of an object), pre-reading can avoid the pressure of small-block IO on the network disk, thereby reducing the pressure on the network and disk in the read scenario, further reducing the number of object accesses, and improving read performance.

[0061] A skip read pre-reading method provided by an embodiment of the present invention may further include:

[0062] After each pre-reading operation and reading operation, the skip reading record information recorded in the skip reading record structure is updated, and the skip reading record information includes a step length, a data length, and a reading end position.

[0063] It should be noted that in order to facilitate the rapid acquisition of corresponding information when determining whether it is a read operation in skip mode, determining the pre-read data position and the pre-read data length, etc., the present application can pre-set a skip record structure, and then record the step size, read end position and data length of each pre-read operation or read operation in the skip record structure, thereby facilitating the rapid implementation of the corresponding steps.

[0064] A skip read pre-reading method provided by an embodiment of the present invention may further include, before updating the skip read record information recorded in the skip read record structure:

[0065] Open the target file, create a handle, and create a skip record structure in the handle;

[0066] The skip-reading method may further include:

[0067] Close the target file and handle, and clear the skip record structure.

[0068] It should be noted that the skip record structure can be created by creating a handle corresponding to the target file when opening the target file, and then creating the skip record structure in the handle, so as to facilitate the storage and query of the skip record structure; when the target file is closed and the handle is closed later, in order to facilitate the subsequent re-implementation of the judgment of the read operation and the pre-reading operation without being affected by the expired information, the embodiment of the present application will also clear the skip record structure. Among them, the handle is an identifier used to identify an object or project, which can be used to describe a window, a file, etc. It is worth noting that the handle cannot be a constant.

[0069] A skip read pre-reading method provided by an embodiment of the present invention may further include, after locally reading data corresponding to a read request:

[0070] Clear the data corresponding to the read request stored locally.

[0071] It should be noted that in order to ensure that the local storage server has sufficient storage space to store unaccessed data, in the embodiment of the present application, after each reading of the local data of the storage server, the accessed data will be cleared, thereby achieving the circular and effective use of the local storage space.

[0072] In a specific application scenario, the overall steps of a skip read pre-reading method provided by an embodiment of the present invention can be as follows: Figure 2 As shown, specifically including:

[0073] 1. After the target file is opened, create a handle;

[0074] 2. When creating a handle, create a skip record structure to record the current skip status; the skip record structure is saved in the handle;

[0075] 3. Update the skip record information in the skip record structure at each read operation, identify the skip mode based on the skip record information, and calculate the skip data position to achieve pre-reading;

[0076] 4. When the file is closed, the handle is closed and the skip record structure is cleared.

[0077] When implementing skip mode recognition, the example diagram can be as follows Figure 3 As shown (wherein the skip information is the skip record information, and the pre-read position is the pre-read data position), specifically, it may include:

[0078] 1. The skip record information may mainly include skip_len (step length), last_read (the end position of the last read operation), last_read_len (the length of the data read by the last read operation or pre-read operation, which can be referred to as the read length), skip_num (number of skips), and readadhead_pos (the end position of the last pre-read operation, that is, the end position of the last pre-read operation);

[0079] 2. After the first read request arrives, update last_read_len to record the current read length;

[0080] 3. After the second read request arrives, skip_len and last_read_len are recorded;

[0081] 4. After the third read request arrives, if skip_len and last_read_len are the same as the last time, the current read mode is considered to be skip read mode, and the skip_num count is updated to 1;

[0082] 5. When the skip_num count is greater than 1, the current read mode is considered to be skip read mode, the pre-read data position is calculated, and pre-reading is initiated.

[0083] When implementing data pre-reading, the example diagram can be as follows Figure 4 As shown, it may specifically include:

[0084] 1. After determining the skip mode, firstly, it is necessary to preliminarily calculate the pre-read data position and pre-read data length, and record the new pre-read information (including readahead_pos), and then determine the pre-read data position as the read end position (readahead_pos) of the last pre-read operation. If it is the first pre-read operation, the pre-read data position is the read end position of the last read operation, and the pre-read length is the read length (last_readahead_len) of the last read operation; if it is a continuous pre-read operation, the pre-read data length is doubled each time by readahead_len (the pre-read data length of the last pre-read operation), with a maximum of 200M and a minimum length of 1 cycle (skip_len+len, len is the data length read in a single read operation), and is aligned to the cycle; after preliminarily calculating the pre-read data position, update readahead_pos (the read end position of the pre-read operation initiated this time) and readahead_len (the pre-read data length this time);

[0085] 2. Pre-read position refinement, using two different pre-read IOs according to the skip mode cycle and object (obj) size:

[0086] 1) When the cycle < object size and on average each object initiates more than two read requests, whole-object prefetching is initiated at this time, that is, the prefetch data position initially calculated in step 1 is the prefetch data position to be initiated this time;

[0087] 2) When the cycle > object size and on average each object initiates less than 1 read request, multi-block skip prefetching is adopted at this time. According to the prefetch data position initially calculated in step 1, the actual positions to be read are selected according to the step size and read length, so that the prefetch position is split into multiple prefetch skip segments.

[0088] 3. Initiate prefetching according to the calculated prefetch data position.

[0089] This application records the characteristics of the recognition read operation (i.e., the IO characteristics) to determine the skip mode, and at the same time performs whole-object prefetching or skip prefetching according to the skip cycle and the underlying object size, so as to improve the prefetching and read performance and reduce the network disk consumption for the small-step skip read scenario.

[0090] The embodiment of the present invention also provides a skip prefetching device, as Figure 5 shown, which may include:

[0091] A reading module 11, configured to: receive a read request, if the data corresponding to the read request has been stored locally, read the data corresponding to the read request locally, otherwise, read the data corresponding to the read request from the target file;

[0092] A first prefetching module 12, configured to: after reading the data corresponding to the read request, determine whether the current read mode is the skip mode. If so, predict the prefetch data position and prefetch data length of the data to be read after the current moment based on the skip mode, prefetch the data corresponding to the prefetch data position and prefetch data length from the target file, and store the prefetched data locally. Otherwise, determine that no prefetching operation is required; wherein, the prefetching operation is an operation of prefetching corresponding data from the target file.

[0093] For a skip prefetching device provided by the embodiment of the present invention, the first prefetching module may include:

[0094] A first determining unit, configured to: if the data lengths and step sizes of the data read in the current read operation and the previous read operation are the same, determine that the current read mode is the skip mode; wherein, the step size of any read operation is the data length between the read start position of the any read operation and the read end position of the previous read operation of the any read operation, and the read operation is an operation of reading the data corresponding to any read request after receiving the any read request.

[0095] For a skip prefetching device provided by the embodiment of the present invention, the first prefetching module may include:

[0096] The second determination unit is used to: if the first pre-read operation is currently required, determine the pre-read data position to be the position after the read end position of the last read operation is offset by a step length, and the pre-read data length is the data length of the last read operation; if the nth pre-read operation among multiple consecutive pre-read operations is currently required, determine the pre-read data position to be the position after the read end position of the last pre-read operation is offset by 1 to n times the step length, and the pre-read data length corresponding to each pre-read data position is the data length of the first pre-read operation among the multiple consecutive pre-read operations.

[0097] A skip-reading pre-reading device provided by an embodiment of the present invention may further include:

[0098] The second pre-reading module is used to: after predicting the pre-read data position of the data to be read after the current moment based on the skip reading mode, if the object used to implement the target file storage is larger than the cycle of the skip reading mode, pre-read each object where the pre-read data position is located in a whole object reading manner, and store the pre-read data locally; otherwise, execute the step of pre-reading the data corresponding to the pre-read data position and the pre-read data length in the target file.

[0099] A skip-reading pre-reading device provided by an embodiment of the present invention may further include:

[0100] The updating module is used to update the skip record information recorded in the skip record structure after each pre-reading operation and each reading operation, wherein the skip record information includes a step length, a data length, and a reading end position.

[0101] A skip-reading pre-reading device provided by an embodiment of the present invention may further include:

[0102] The opening module is used to: before updating the skip record information recorded in the skip record structure, open the target file, create a handle, and create a skip record structure in the handle;

[0103] The closing module is used to: close the target file and handle, and clear the skip record structure.

[0104] A skip-reading pre-reading device provided by an embodiment of the present invention may further include:

[0105] The clearing module is used to clear the data corresponding to the read request stored locally after reading the data corresponding to the read request locally.

[0106] The embodiment of the present invention further provides a skip read pre-reading device, which may include:

[0107] Memory for storing computer programs;

[0108] A processor is used to implement the steps of any of the above skip read and pre-read methods when executing a computer program.

[0109] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above skip-reading pre-reading methods can be implemented.

[0110] It should be noted that the description of the relevant parts of the skip read-ahead device, equipment and storage medium provided by the embodiment of the present invention can refer to the detailed description of the corresponding parts in the skip read-ahead method provided by the embodiment of the present invention, which will not be repeated here. In addition, the parts of the above technical solutions provided by the embodiment of the present invention that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0111] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A skip read and pre-read method, characterized in that: include: Receive a read request, and if the data corresponding to the read request is already stored locally, read the data corresponding to the read request locally; otherwise, read the data corresponding to the read request from the target file; After reading the data corresponding to the read request, determining whether the current read mode is a skip read mode, if yes, predicting the pre-read data position and pre-read data length of the data to be read after the current moment based on the skip read mode, pre-reading the data corresponding to the pre-read data position and the pre-read data length from the target file, and storing the pre-read data locally, otherwise, determining that no pre-read operation is required; wherein the pre-read operation is an operation of pre-reading corresponding data from the target file; Wherein, determining whether the current read mode is a skip read mode includes: if the data length and step length of the data read by the current read operation and the previous read operation are the same, determining that the current read mode is a skip read mode; wherein the step length of any read operation is the data length between the read start position of the any read operation and the read end position of the previous read operation of the any read operation, and the read operation is an operation of reading data corresponding to any read request after receiving the any read request; Predicting the pre-read data position and pre-read data length of data to be read after the current moment based on the skip read mode includes: if the first pre-read operation is currently required, determining the pre-read data position to be the position after the read end position of the last read operation is offset by the step length, and the pre-read data length is the data length of the last read operation; if the nth pre-read operation is currently required among multiple consecutive pre-read operations, determining the pre-read data position to be the position after the read end position of the last pre-read operation is offset by 1 to n times the step length, and the pre-read data length corresponding to each pre-read data position is the data length of the first pre-read operation among the multiple consecutive pre-read operations; the first pre-read operation is the first pre-read operation when no pre-read operation has been performed before the current moment, or the first pre-read operation when there is a read operation between the last pre-read operation and the current pre-read operation when the pre-read operation is not performed continuously; After predicting the pre-read data position of the data to be read after the current moment based on the skip mode, it also includes: if the object used to implement the storage of the target file is larger than the cycle of the skip mode, pre-read each object where the pre-read data position is located in a whole object reading manner, and store the pre-read data locally; otherwise, execute the step of pre-reading the data corresponding to the pre-read data position and the pre-read data length in the target file; wherein one cycle of the skip mode is the sum of the data length and the step length read in a certain read operation.

2. The method according to claim 1, characterized in that Also includes: After each pre-reading operation and each reading operation, the skip reading record information recorded in the skip reading record structure is updated, and the skip reading record information includes a step length, a data length, and a reading end position.

3. The method according to claim 2, characterized in that Before updating the skip record information recorded in the skip record structure, it also includes: Opening the target file, creating a handle, and creating the skip record structure in the handle; the method further includes: The target file and the handle are closed, and the skip record structure is cleared.

4. The method according to claim 3, characterized in that After the data corresponding to the read request is read locally, the method further includes: The data corresponding to the read request stored locally is cleared.

5. A skip-reading pre-reading device, characterized in that: include: The reading module is used to: receive a read request, and if the data corresponding to the read request is already stored locally, read the data corresponding to the read request locally; otherwise, read the data corresponding to the read request from a target file; A first pre-reading module is used to: after reading the data corresponding to the read request, determine whether the current read mode is a skip read mode, if yes, predict the pre-read data position and pre-read data length of the data to be read after the current moment based on the skip read mode, pre-read the data corresponding to the pre-read data position and the pre-read data length from the target file, and store the pre-read data locally; otherwise, determine that no pre-read operation is required; wherein the pre-read operation is an operation of pre-reading corresponding data from the target file; Wherein, the first pre-reading module includes: A first determination unit is used to: if the data length and step length of the data read by the current read operation and the previous read operation are the same, determine that the current read mode is a skip read mode; wherein the step length of any read operation is the data length between the read start position of the any read operation and the read end position of the previous read operation of the any read operation, and the read operation is an operation of reading data corresponding to any read request after receiving the any read request; The second determining unit is used for: if the first pre-reading operation is required to be performed currently, determining the pre-reading data position to be the position after the read end position of the last read operation is offset by the step length, and the pre-reading data length is the data length of the last read operation; if the nth pre-reading operation is required to be performed currently among multiple consecutive pre-reading operations, determining the pre-reading data position to be the position after the read end position of the last pre-reading operation is offset by 1 to n times the step length, and the pre-reading data length corresponding to each pre-reading data position is the data length of the first pre-reading operation among the multiple consecutive pre-reading operations; the first pre-reading operation is the first pre-reading operation when no pre-reading operation has been performed before the current moment, or the first pre-reading operation when there is a read operation between the last pre-reading operation and the current pre-reading operation when the pre-reading operation is not performed continuously; The skip-reading pre-reading device also includes: The second pre-reading module is used for: after predicting the pre-read data position of the data to be read after the current moment based on the skip reading mode, if the object used to implement the storage of the target file is larger than the cycle of the skip reading mode, pre-reading each object where the pre-read data position is located in a whole object reading manner, and storing the pre-read data locally; otherwise, executing the step of pre-reading the data corresponding to the pre-read data position and the pre-read data length in the target file; wherein one cycle of the skip reading mode is the sum of the data length and the step length read in a certain read operation.

6. A skip-reading and pre-reading device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the skip read-ahead method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the skip-reading pre-reading method according to any one of claims 1 to 4 are implemented.

Images