Data loading method and device, storage medium and electronic equipment
By receiving read requests and determining the preload interval based on preload evaluation information, the disk data is loaded into memory, which solves the problem of slow application access speed and improves the operation efficiency of application.
Patent Information
- Application Number
- CN202310838560.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-07
- Publication Date
- 2025-01-07
AI Technical Summary
The prior art is difficult to effectively improve the speed of applications accessing disk data, resulting in limited application running speed.
By receiving a read request, preload evaluation information is obtained. If the information meets the conditions, the preload interval is determined and the preload data on disk is loaded into memory to prepare the data that may be accessed in advance.
It significantly improves the speed of applications accessing data, reduces the waiting time for synchronization requests to disk, and improves the running speed of applications.
Smart Images
Figure CN119271290A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of electronic technology, and in particular, relates to a data loading method, device, computer-readable storage medium and electronic device. Background Art
[0002] With the development of information technology and the improvement of productivity, the demand of applications for file system IO performance is also increasing. The operation of applications is usually inseparable from the access to disk data. The faster the speed of accessing disk data, the faster the application runs. Therefore, in order to increase the running speed of applications, it is necessary to increase the speed of application access to disk data. Summary of the invention
[0003] The embodiments of the present application provide a data loading method, device, storage medium and electronic device, which can improve the speed at which an application accesses corresponding data.
[0004] In a first aspect, an embodiment of the present application provides a data loading method, comprising:
[0005] receiving a first read request;
[0006] Acquire information to be evaluated corresponding to the first read request and used for performing preloading evaluation;
[0007] If the information to be evaluated meets the preloading condition, determining a first preloading interval according to a first data interval in which the data requested to be read by the first read request is located, and the amount of preloaded data in the first preloading interval is greater than the target data amount;
[0008] Load the preloaded data from disk into memory
[0009] In a second aspect, an embodiment of the present application provides a data loading device, including:
[0010] A request receiving module, configured to receive a first read request;
[0011] An information acquisition module, used to acquire information to be evaluated corresponding to the first read request and used for performing preloading evaluation;
[0012] an interval determination module, configured to determine a first preloading interval according to a first data interval in which the data requested to be read by the first read request is located if the information to be evaluated meets the preloading condition, and the amount of preloaded data in the first preloading interval is greater than a target data amount;
[0013] The data loading module is used to load the preloaded data in the disk into the memory.
[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed on a computer, the computer executes the data loading method provided by the embodiment of the present application.
[0015] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the processor is used to execute the data loading method provided in the embodiment of the present application by calling a computer program stored in the memory.
[0016] In an embodiment of the present application, when the information to be evaluated for preloading evaluation corresponding to the first read request meets the preloading condition, the first preloading interval is determined according to the first data interval in which the data requested to be read by the first read request is located; the preloaded data in the disk that is in the first preloading interval is loaded into the memory, and the data amount of the preloaded data is greater than the target data amount. When it is determined that preloading can be performed currently, the data that the application may access, that is, the preloaded data, can be loaded from the disk to the memory in advance, which can improve the speed of the application accessing the corresponding data, that is, the preloaded data. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The technical solution and beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.
[0018] Figure 1 This is a first flow chart of the data loading method provided in the embodiment of the present application.
[0019] Figure 2 This is a second flow chart of the data loading method provided in the embodiment of the present application.
[0020] Figure 3 It is a scenario diagram of the data loading method provided in an embodiment of the present application.
[0021] Figure 4 It is a structural diagram of a data loading device provided in an embodiment of the present application.
[0022] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0023] It should be noted that the terms "first", "second" and "third" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but some embodiments also include steps or modules that are not listed, or some embodiments also include other steps or modules inherent to these processes, methods, products or devices.
[0024] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0025] The embodiments of the present application provide a data loading method, a data loading device, a storage medium and an electronic device. The execution subject of the data loading method can be the data loading device provided in the embodiments of the present application, or an electronic device integrating the data loading device, wherein the data loading device can be implemented in hardware or software. Among them, the electronic device can be a device such as a smart phone, a tablet computer, a PDA, a laptop computer, etc. that is equipped with a processor and has data loading capabilities.
[0026] See also Figure 1 , Figure 1 This is a schematic diagram of the first process of the data loading method provided in the embodiment of the present application. The process may include:
[0027] In 101 , a first read request is received.
[0028] The first read request may be initiated by the application to the file system to request the file system to access corresponding data, so that the file system receives the first read request.
[0029] After receiving the first read request, the file system can detect whether the data requested by the first read request exists in the memory; if the data requested by the first read request exists in the memory, the file system can obtain the data requested by the first read request from the memory and return it to the application; if the data requested by the first read request does not exist in the memory, the file system can obtain the data requested by the first read request from a disk, such as a local disk or a network disk, and return it to the application.
[0030] In 102, to-be-evaluated information corresponding to the first read request and used for preloading evaluation is obtained.
[0031] It is understandable that retrieving data from the memory and returning it to the application is faster than retrieving data from the disk and returning it to the application. Therefore, in order to increase the speed at which the application reads data, the information to be evaluated corresponding to the first read request for preloading evaluation can be obtained, so that when it is determined that the information to be evaluated meets the preloading conditions, the data that is not currently accessed by the application but may be accessed in the future can be loaded from the disk into the memory in advance, so that when the application needs to access the data, the data can be directly retrieved from the memory and returned to the application, thereby increasing the speed at which the application accesses the data.
[0032] The information to be evaluated for performing the preloading evaluation may include information related to the first read request, and may also include information related to a second read request, a third read request, etc., received before receiving the first read request.
[0033] Among them, among the read requests received before receiving the first read request, the difference between the receiving time of the read requests used to determine the information to be evaluated, such as the second read request, the third read request, etc. and the receiving time of the first read request is less than the difference between the receiving time of other read requests and the receiving time of the first read request.
[0034] For example, assuming that the file system receives the sixth read request, the fifth read request, the third read request, the second read request and the first read request in sequence, the file system can determine the information to be evaluated based on the first read request, the second read request and the third read request; the file system can also determine the information to be evaluated based on the first read request, the second read request, the third read request and the fifth read request.
[0035] In 103, if the information to be evaluated meets the preloading condition, a first preloading interval is determined according to a first data interval where the data requested to be read by the first read request is located, and the amount of preloaded data in the first preloading interval is greater than the target data amount.
[0036] Among them, the data is stored in the disk in the form of files, and the data requested by the first read request may be data in a certain interval in a certain file. Therefore, the first read request will also carry the file identifier, starting position and size of the file where the requested data is located. Based on the file identifier, the file system can determine the file to which the data requested by the first read request belongs, and based on the starting position and size, the file system can determine the first data interval where the data of the first read request is located.
[0037] For example, assuming that the data requested by the first read request is a segment of data in a file whose file size is 1024 bytes, the starting position of the data read this time is the 200th byte position, and the size is 20 bytes, then the first data interval is [200,219].
[0038] It can be understood that the information related to the first read request and the information related to the second read request, the third read request and other read requests received before receiving the first read request reflect the file reading behavior of the application to a certain extent. If it is determined based on the file reading behavior that the application is regularly accessing data, such as the interval where the data requested to be read by the first read request, the second read request and the third read request is located is a continuous interval in a certain file, then it can be predicted that the data that the application may access in the future is also an interval in the file that has not been accessed and is continuous with the interval where the data requested to be read by the first read request, the second read request and the third read request is located. Therefore, the information to be evaluated that can reflect the behavior of the application in regularly accessing data can be determined as the information to be evaluated that meets the preloading condition.
[0039] After obtaining the information to be evaluated, it can be determined whether the information to be evaluated meets the preloading condition. If the information to be evaluated meets the preloading condition, the first data interval where the data requested to be read by the first read request is located can be obtained, and then the first preloading interval is determined based on the first data interval. The starting position of the first preloading interval is the starting position of the first data interval + the size of the data in the first data interval, and the amount of data in the first preloading interval is greater than the target data amount.
[0040] The target data volume may be set by a user or by the electronic device based on certain rules.
[0041] In an optional embodiment, a corresponding pre-reading algorithm is provided in the related art, such as heuristic loading: the pre-reading algorithm in the Linux kernel and the client pre-reading algorithm in CephFS both adopt this form, and the file system reads the data from the disk to the memory in advance according to the current reading behavior of the application. The amount of data in the first pre-loading interval of the present application can be greater than the amount of data to be read by the application next determined by the pre-reading algorithm, such as 4-16 times the amount of data to be read by the application next determined by the pre-reading algorithm.
[0042] Since the process of the file system reading data from the disk is: sending a read request to the disk -> waiting for data to be returned -> returning the read result, when the amount of data in the first preloading interval determined by the present application is 4 times the amount of data to be read next by the application determined by the pre-reading algorithm in the related art, the present application only needs to send a read request to the disk once, while the pre-reading algorithm in the related art needs to send multiple read requests to the disk. Therefore, it can be seen that the present application can reduce the waiting time for synchronously requesting data from the disk.
[0043] In 104, the preloaded data in the disk is loaded into the memory.
[0044] It is understandable that after determining the first preload interval, the file system can send a read request to the disk so that the disk can load the preload data in the first preload interval from the disk into the memory, such as the cache. When the data requested to be read by the read request sent by the application is in the first preload interval, the file system can directly obtain the corresponding data from the memory and return it to the application. Compared with the solution of obtaining the corresponding data from the disk and returning it to the application, the speed of the application accessing data can be improved, thereby improving the running speed of the application.
[0045] In this embodiment, when the information to be evaluated for preloading evaluation corresponding to the first read request meets the preloading condition, the first preloading interval is determined according to the first data interval in which the data requested to be read by the first read request is located; the preloaded data in the first preloading interval on the disk is loaded into the memory, and the data amount of the preloaded data is greater than the target data amount, so that the data that is predicted to be accessed by the application, that is, the preloaded data, can be loaded from the disk to the memory in advance, which can improve the speed of the application accessing the corresponding data, that is, the preloaded data.
[0046] In an optional embodiment, obtaining information to be evaluated corresponding to the read request and used for performing preloading evaluation includes:
[0047] Obtaining a first data interval where data requested to be read by a first read request is located;
[0048] Acquire a second data interval where data requested to be read by a second read request received before receiving the first read request is located;
[0049] Acquire a third data interval where data requested to be read by a third read request received before receiving the second read request is located;
[0050] The first data interval, the second data interval and the third data interval are determined as information to be evaluated.
[0051] In this embodiment, the first data interval where the data requested to be read by the first read request is located, the third data interval where the data requested to be read by the second read request is located, and the third data interval where the data requested to be read by the third read request is located can be determined as the information to be evaluated.
[0052] In an optional embodiment, if the current scene is not a preloading scene, the first data interval and the second data interval meet the conditions for triggering pre-reading, and the second data interval and the third data interval meet the conditions for triggering pre-reading, then it is determined that the information to be evaluated meets the preloading conditions.
[0053] The preloaded scene is a scene in which data in the second preloaded interval corresponding to the preloaded scene in the disk is loaded into the memory. In other words, assuming that the scene is currently loading data in the corresponding preloaded interval into the memory, the current scene is determined to be the preloaded scene.
[0054] The condition for triggering pre-reading may be that the data intervals where the data requested to be read by the two read requests are located are continuous data intervals.
[0055] In an optional embodiment, when pre-reading has been triggered before, the end position of the data interval where the data requested to be read by the subsequent read request, that is, the end of the interval, must exceed the end position of the pre-reading interval when the pre-reading was triggered before, that is, the end of the interval can continue to trigger pre-reading.
[0056] In an optional embodiment, the condition for triggering pre-reading may also be that the first interval of the data interval requested to be read by the two read requests is within a certain range.
[0057] For example, assuming that the current scene is not a preloading scene, pre-reading has not been triggered before, that is, there is no data interval that meets the condition for triggering pre-reading, and the first data interval, the second data interval, and the third data interval are data intervals of a certain file (19,39], (9,19], and [0,9], respectively, then it can be determined that the information to be evaluated meets the preloading condition.
[0058] In an optional embodiment, before determining the first data interval, the second data interval, the third data interval and the first virtual pre-read interval as the information to be evaluated, the method further includes:
[0059] If the current scene is a preload scene, a second preload interval corresponding to the preload scene is obtained, and a first virtual pre-read interval is determined according to the first data interval. The preload scene is a scene in which data in the second preload interval in the disk is loaded into the memory.
[0060] Determining the first data interval, the second data interval, and the third data interval as information to be evaluated includes:
[0061] The first data interval, the second data interval, the third data interval, the second pre-loading interval and the first virtual pre-reading interval are determined as information to be evaluated.
[0062] Among them, the virtual pre-read interval is the virtual pre-read interval calculated when it is assumed that the data interval of the data requested to be read by the current read request and the data interval of the data requested to be read by the read request received before the current read request and whose reception time is closest to the reception time of the current read request meet the conditions for triggering pre-reading.
[0063] The starting position of the first virtual pre-read interval, i.e., the beginning of the interval, is the ending position of the first data interval, i.e., the end of the interval, and the size is usually the smaller value of twice the size of the first data interval, the maximum limit of the pre-read interval size (usually 8MB), the size of the file where the first virtual pre-read interval is located, and the end position of the first data interval, i.e., the end of the interval. The size of the first pre-read interval is twice the size of the data interval where the data requested to be read by the read request is located.
[0064] For example, suppose that the file system receives a read request for requesting data in a certain file, and the read request is the first read request for data in the file, the file size is 1024, and the starting position of the data interval of the data requested to be read by the read request is 200, and the size is 20. Then, the data interval of the data to be read by the read request is [200,219], and the virtual pre-read interval is (219,259].
[0065] In an optional embodiment, if the first virtual pre-read interval is not in the second pre-load interval, the first data interval and the second data interval meet the conditions for triggering pre-reading, and the second data interval and the third data interval meet the conditions for triggering pre-reading, it is determined that the information to be evaluated meets the pre-load condition.
[0066] For example, assuming that the current scenario is a preloading scenario, the first data interval, the second data interval, the third data interval, the first virtual pre-reading interval, and the second preloading interval are respectively the data intervals (189, 209], (179, 189], (169, 179], (209, 249], and (39, 199] of a certain file. It can be determined that the first virtual pre-reading interval (209, 249] is not in the second preloading interval (39, 199], the first data interval (189, 209] and the second data interval (179, 189] meet the conditions for triggering pre-reading, and the second data interval (179, 189] and the third data interval (169, 179] also meet the conditions for triggering pre-reading. Therefore, it is determined that the information to be evaluated corresponding to the first read request meets the preloading condition.
[0067] In an optional embodiment, if the first virtual pre-read interval is in the second pre-load interval, and the first data interval and the second data interval do not meet the conditions for triggering pre-reading, the loading of data in the disk that is in the second pre-load interval but not loaded into the memory is stopped.
[0068] For example, assuming that the first data interval, the second data interval, the first virtual pre-read interval and the second pre-load interval are the data intervals (169, 179], (39, 79], (179, 199], (39, 199] of a certain file respectively, it can be determined that the first virtual pre-read interval is in the second pre-load interval (39, 199], and the first data interval (169, 179] and the second data interval (39, 79] do not meet the triggering condition for pre-reading, then the loading of the pre-loaded data in the second pre-load interval (39, 199] on the disk that has not been loaded into the memory can be stopped to prevent loading of useless data.
[0069] In an optional embodiment, determining the first preloading interval according to the first data interval where the data requested to be read by the first read request is located includes:
[0070] Determine a first virtual pre-reading interval according to the first data interval;
[0071] A first preloading interval is determined according to the first virtual prereading interval, a starting position of the first preloading interval is the same as a starting position of the first virtual prereading interval, and a size of the first preloading interval is larger than a size of the first virtual prereading interval.
[0072] For example, the size of the first preload interval may be 4-16 times the size of the first virtual pre-read interval. That is, the data volume corresponding to 4-16 times the size of the first virtual pre-read interval is the target data volume.
[0073] For example, assuming that the data interval of the data requested to be read by the read request is (189, 209], the virtual pre-read interval can be determined to be (209, 249], and then the pre-load interval can be determined to be (209, 369].
[0074] In an optional embodiment, loading the preloaded data in the disk into the memory includes:
[0075] Dividing the first preloading interval into a plurality of consecutive sub-preloading intervals;
[0076] Load the preload data in each sub-preload interval in the disk into the memory in sequence;
[0077] During the process of loading the preload data in the corresponding sub-preload interval, if a fourth read request is received, the fourth read request is processed after the preload data in the corresponding sub-preload interval is loaded completely.
[0078] For example, when the file system loads the preloaded data in the first preload interval (209, 369] in the disk into the memory, the first preload interval can be divided into a plurality of sub-preload intervals, such as the sub-preload intervals (209, 219], (219, 229], (229, 239], (239, 249] ... (339, 349], (349, 359], (359, 369], and then the preloaded data in the sub-preload intervals (209, 219], (219, 229], (229, 239], (239, 249] ... (339, 349], (349, 359], (359, 369] are loaded in sequence. In the process of loading the preloaded data in the sub-preload interval (239, 249], if the file system receives a read request, The fourth read request requests to read the data in the data interval (239, 249], and the fourth read request may be blocked first, and after the preloaded data in the sub-preload interval (239, 249] is loaded, the data in the data interval (239, 249] is obtained from the memory and returned to the application; in the process of loading the preloaded data in the sub-preload interval (239, 249], if the file system receives a fourth read request, and the fourth read request requests to read the data in the data interval (349, 359] or in the data interval (379, 389], the fourth read request may be blocked first, and after the preloaded data in the sub-preload interval (239, 249] is loaded, the data in the data interval (349, 359] or in the data interval (379, 389] is obtained and returned to the application.
[0079] It should be noted that by adjusting the triggering timing and interval parameters for loading preloaded data, the data loading method provided in the embodiment of the present application can also be applied to block storage and object storage systems. The specific adjustment can be determined according to the characteristics of the storage system.
[0080] For example, if the object storage stores objects of 4 MB in size and there are many continuous read behaviors, that is, there are many read requests that meet the conditions for triggering pre-reading, the maximum value of the pre-load interval can be adjusted to 4 MB according to the size of the object.
[0081] See also Figure 2 , Figure 2 This is a second flow chart of the data loading method provided in the embodiment of the present application, and the flow chart may include:
[0082] In 201 , a first read request is received.
[0083] In 202, a first data interval where data requested to be read by a first read request is located is obtained.
[0084] In 203 , a first virtual pre-read interval is determined according to a first data interval where the data requested to be read by the first read request is located.
[0085] In 204 , a second data interval in which data requested to be read by a second read request received before the first read request is received is obtained.
[0086] In 205 , a third data interval in which data requested to be read by a third read request received before the second read request is received is obtained.
[0087] In 206 , if the current scene is a preloaded scene, a second preloaded interval corresponding to the preloaded scene is obtained, and the preloaded scene is a scene in which data in the second preloaded interval in the disk is loaded into the memory.
[0088] In 207 , if the first virtual pre-read interval is in the second pre-load interval, and the first data interval and the second data interval do not satisfy the condition for triggering pre-reading, the loading of the data in the second pre-load interval in the disk that has not been loaded into the memory is stopped.
[0089] In 208, if the first virtual pre-read interval is not in the second pre-loading interval, the first data interval and the second data interval meet the conditions for triggering pre-reading, and the second data interval and the third data interval do not meet the conditions for triggering pre-reading, then the first pre-loading interval is determined according to the first virtual pre-reading interval, the starting position of the first pre-loading interval is the same as the starting position of the first virtual pre-reading interval, and the size of the first pre-loading interval is larger than the size of the first virtual pre-reading interval.
[0090] In 209, the preloaded data in the disk is loaded into the memory.
[0091] It should be noted that the specific implementation examples of steps 201 to 209 can be found in other implementation examples and will not be described in detail here.
[0092] See also Figure 3 , the data loading method provided in the embodiment of the present application will be further explained below by taking an application accessing a file whose data interval is [0,1024] and no data in the file is currently stored in the memory as an example.
[0093] At time t1, the file system receives a read request R1. The data interval A11 requested by the read request R1 is the data interval [0,9]. The file system can obtain the data in the data interval [0,9] from the disk and return it to the application. After that, the lazy notification loading thread receives a detection request. After receiving the detection request, the lazy notification loading thread first calculates the virtual pre-read interval A21 as the data interval (9,29) according to the data interval [0,9]. Since the read request R1 is the first read request received by the file system to request to read the data in the file, it can be determined that the current scene is not in the lazy notification loading scene, or it can be said that the virtual pre-read interval (9,29] is not in the preloading interval, and the information to be evaluated corresponding to the read request R1, that is, the data interval [0,9] does not meet the preloading condition. Then, the subsequent detection of whether the information to be evaluated meets the preloading condition can be continued. Among them, the lazy notification loading thread is a resident thread. Each time the read thread receives the read data, it will send a check request to the blocking message queue, and the lazy notification loading thread will take out the check request from the queue for processing.
[0094] At time t2, the file system receives a read request R2. The data interval A12 requested by the read request R2 is the data interval (9,19]. The file system can obtain the data in the data interval [10,19] from the disk and return it to the application. After that, the lazy notification loading thread receives a detection request. After receiving the detection request, the lazy notification loading thread first calculates the virtual pre-read interval A22 as the data interval (19,39) based on the data interval (9,19]. It can be determined that at this time, the current scene is not a pre-loaded scene either, or the virtual pre-read interval (19,39] is not in the pre-loaded interval. Since the data intervals [0,9] and (9, 19] is a continuous data interval, it can be determined that the data intervals [0,9] and (9,19] meet the condition for triggering pre-reading, so it can be determined that the pre-reading interval A31 is (19,39], then the file system can send a read request to the disk to load the data in the pre-reading interval (19,39] on the disk into the memory. However, since only the data intervals [0,9] and (9,19] meet the condition for triggering pre-reading, it can be determined that the information to be evaluated corresponding to the read request R2, that is, the data intervals [0,9] and (9,19], do not meet the pre-loading condition, then, it can continue to detect whether the information to be evaluated obtained subsequently meets the pre-loading condition.
[0095] At time t3, the file system receives a read request R3, and the data interval A13 requested to be read by the read request R3 is the data interval (19,39]. Then, if the data in the pre-read interval (19,39] on the disk has been loaded into the memory at this time, the data in the data interval (19,39] can be directly obtained from the memory and returned to the application. After returning the data in the data interval (19,39] to the application, the lazy notification loading thread receives a detection request. After receiving the detection request, the lazy notification loading thread first calculates the virtual pre-read interval A23 as the data interval (39,79] based on the data interval (19,39]. It can be determined that at this time, the current scene is not a pre-loaded scene either, or the virtual pre-read interval (39,79] is not in the pre-loaded interval. Since the data intervals (9,19] and (19,39 ] is a continuous data interval, it can be determined that the data intervals (9,19] and (19,39] meet the conditions for triggering pre-reading, so it can be determined that the pre-reading interval A32 is the data interval (39,79], then the file system can send a read request to the disk to load the data in the pre-reading interval (39,79] on the disk into the memory. Since the data intervals [0,9] and (9,19] meet the conditions for triggering pre-reading, and the data intervals (9,19] and (19,39] meet the conditions for triggering pre-reading, it is possible to determine the information to be evaluated corresponding to the read request R3, that is, the data intervals [0,9], (9,19] and (19,39] meet the preloading conditions, and the preloading interval A41 is (39,199], then the file system can load the preloaded data in the preloading interval (39,199] on the disk into the memory.
[0096] In the process of loading the preloaded data in the preload interval (39, 199] on the disk into the memory, that is, when the current scene is the preload scene, at time t4, the file system receives a read request R4, and the data interval A14 requested to be read by the read request R4 is the data interval (39, 79]. Then, if the data in the data interval (39, 79] on the disk has been loaded into the memory at this time, the data in the data interval (39, 79] can be directly obtained from the memory and returned to the application. When the data in the data interval (39, 79] is returned to the application, After returning to the application, the lazy notification loading thread receives the detection request. After receiving the detection request, the lazy notification loading thread first calculates the virtual pre-read interval A24 as the data interval (79,159] based on the data interval (39,79]. Since this is a pre-loading scenario, and the virtual pre-read interval (79,159] is in the pre-loading interval (39,199], and the data intervals (39,79] and (19,39] meet the conditions for triggering pre-reading, the pre-loaded data in the pre-loading interval (39,199] on the disk can continue to be loaded into the memory.
[0097] In the process of loading the preloaded data in the preload interval (39,199] on the disk into the memory, that is, when the current scene is the preload scene, at time t5, the file system receives a read request R5, and the data interval A15 of the data requested to be read by the read request R5 is (169,179]. Then, if the data in the data interval (169,179] on the disk has been loaded into the memory at this time, the data in the data interval (169,179] can be directly obtained from the memory and returned to the application. After returning the data in the data interval (169,179] to the application, the lazy notification loading thread receives the detection request After receiving the detection request, the lazy notification loading thread first calculates the virtual pre-read interval A25 as the data interval (179, 199] according to the data interval (169, 179]. It can be determined that the virtual pre-read interval (179, 199] is in the preload interval (39, 199]. However, since the data intervals (169, 179] and (39, 79] are not continuous data intervals, it can be determined that the data intervals (179, 199] and (39, 79] do not meet the triggering pre-read condition, and the loading of the preloaded data in the disk that is in the preload interval (39, 199] and has not been loaded into the memory can be stopped to prevent loading of useless data.
[0098] At time t6, the file system receives a read request R6. The data interval A16 of the data requested to be read by the read request R6 is (179, 189]. Therefore, the data in the data interval (179, 189) can be obtained and returned to the application. After returning the data in the data interval (179, 189) to the application, the lazy notification loading thread receives a detection request. After receiving the detection request, the lazy notification loading thread first calculates the virtual pre-read interval A26 as the data interval (189, 209) according to the data interval (179, 189), which is not in the preload interval (39, 199). , it can also be determined that the data intervals (179,189] and (169,179] meet the conditions for triggering pre-reading, but since the data intervals (169,179] and (39,79] do not meet the conditions for triggering pre-reading, it is possible to determine the information to be evaluated corresponding to the read request R6, that is, the data intervals (179,189], (169,179] and (39,79] do not meet the pre-loading conditions, then a normal pre-reading can be performed, that is, a read request is sent to the disk to load the data in the pre-reading interval A33(189,209] determined according to the data interval (179,189] on the disk into the memory.
[0099] At time t7, the file system receives a read request R7. The data interval A17 of the data requested to be read by the read request R7 is (189, 209]. The data in the data interval (189, 209) can be obtained and returned to the application. After returning the data in the data interval (189, 209) to the application, the lazy notification loading thread receives a detection request. After receiving the detection request, the lazy notification loading thread first calculates the virtual pre-read interval A27 as the data interval (209, 249) according to the data interval (189, 209), which is not in the preload interval (39, 1 99], the data intervals (189, 209] and (179, 189] meet the condition for triggering pre-reading, and since the data intervals (179, 189] and (169, 179] also meet the condition for triggering pre-reading, the information to be evaluated corresponding to the read request R7 is determined, that is, the data intervals (189, 209], (179, 189] and (169, 179] meet the preloading condition, then it can be determined that the preloading interval A32 is (209, 369], and the file system can load the preloaded data in the preloading interval (209, 369] on the disk into the memory.
[0100] When the file system loads the preloaded data in the preload interval (209,369] on the disk into the memory, the preload interval can be divided into multiple sub-preload intervals, such as the sub-preload intervals (209,219], (219,229], (229,239], (239,249]...(339,349], (349,359], (359,369], and in the process of loading the preloaded data in the sub-preload interval (239,249], if the file system receives a read request R8, and the read request R8 requests to read the data in the data interval (239,249], then the read request R8 can be blocked first, and the sub-preload intervals can be blocked. After the preloaded data in the sub-preload interval (239, 249] is loaded, the data in the data interval (239, 249] is obtained from the memory and returned to the application; in the process of loading the preloaded data in the sub-preload interval (239, 249], if the file system receives a read request R9, and the read request R9 requests to read the data in the data interval (349, 359] or in the data interval (379, 389], the read request R9 can be blocked first, and after the preloaded data in the sub-preload interval (239, 249] is loaded, the data in the data interval (349, 359] or in the data interval (379, 389] is obtained and returned to the application.
[0101] When the file system loads the preloaded data in the preload interval (209, 369] on the disk into the memory, that is, when the current scenario is the preload scenario, if the virtual pre-read interval determined according to a read request received at a certain time, such as the data interval where the data requested to be read by the read request R10 is located, is not in the preload interval (209, 369], the data interval where the data requested to be read by the read request R10 is located and the data interval where the data requested to be read by the read request R11, which is received before the read request R10 and whose receiving time is closest to the receiving time of the read request R10, meets the condition for triggering pre-reading, and the data interval where the data requested to be read by the read request R11 is located and the data interval where the data requested to be read by the read request R12, which is received before the read request R11 and whose receiving time is closest to the receiving time of the read request R11, meets the condition for triggering pre-reading, then a new pre-load interval can be further determined, and the data in the disk in the new pre-load interval can be loaded into the memory.
[0102] See also Figure 4 , Figure 4 The data loading device 300 includes: a request receiving module 301 , an information obtaining module 302 , an interval determining module 303 and a data loading module 304 .
[0103] The request receiving module 301 is used to receive a first read request.
[0104] The information acquisition module 302 is used to acquire information to be evaluated corresponding to the first read request and used for performing preloading evaluation.
[0105] The interval determination module 303 is used to determine a first preloading interval according to a first data interval where the data requested to be read by the first read request is located if the information to be evaluated meets the preloading condition, and the amount of preloaded data in the first preloading interval is greater than the target data amount.
[0106] The data loading module 304 is used to load the pre-loaded data in the disk into the memory.
[0107] In an optional embodiment, the information acquisition module 302 can be used to: obtain a first data interval in which data requested to be read by the first read request is located; obtain a second data interval in which data requested to be read by a second read request received before receiving the first read request is located; obtain a third data interval in which data requested to be read by a third read request received before receiving the second read request is located; and determine the first data interval, the second data interval and the third data interval as the information to be evaluated.
[0108] In an optional embodiment, the information acquisition module 302 can be used to: if the current scene is a preloading scene, obtain the second preloading interval corresponding to the preloading scene, and determine the first virtual pre-reading interval based on the first data interval, and the preloading scene is a scene in which the data in the second preloading interval on the disk is loaded into the memory; determine the first data interval, the second data interval, the third data interval, the second preloading interval and the first virtual pre-reading interval as the information to be evaluated.
[0109] In an optional embodiment, the interval determination module 303 can be used to: if the first virtual pre-read interval is not in the second pre-loading interval, the first data interval and the second data interval meet the conditions for triggering pre-reading, and the second data interval and the third data interval meet the conditions for triggering pre-reading, then determine that the information to be evaluated meets the pre-loading conditions.
[0110] In an optional embodiment, the data loading module 304 can be used to: if the first virtual pre-read interval is in the second pre-loading interval, and the first data interval and the second data interval do not meet the conditions for triggering pre-reading, then stop loading the data in the disk that is in the second pre-loading interval and has not been loaded into the memory.
[0111] In an optional embodiment, the interval determination module 303 can be used to: determine a first virtual pre-read interval based on the first data interval; determine the first preloading interval based on the first virtual pre-read interval, the starting position of the first preloading interval is the same as the starting position of the first virtual pre-read interval, and the size of the first preloading interval is larger than the size of the first virtual pre-read interval.
[0112] In an optional embodiment, the data loading module 304 can be used to: divide the first preloading interval into multiple continuous sub-preloading intervals; load the preloading data in each of the sub-preloading intervals in the disk into the memory in turn; in the process of loading the preloading data in the corresponding sub-preloading interval, if a fourth read request is received, the fourth read request is processed after the preloading data in the corresponding sub-preloading interval is loaded.
[0113] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed on a computer, the computer executes the data loading method provided in this embodiment.
[0114] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the processor is used to execute the data loading method provided in the present embodiment by calling a computer program stored in the memory.
[0115] For example, the electronic device may be a mobile terminal such as a tablet computer or a smart phone. Figure 5 , Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0116] The electronic device 400 may include components such as a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the electronic device 400 may also include a screen.
[0117] The processor 401 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing applications stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the electronic device as a whole.
[0118] The memory 402 may be used to store applications and data. The applications stored in the memory 402 include executable codes. The applications may be composed of various functional modules. The processor 401 executes various functional applications and data processing by running the applications stored in the memory 402.
[0119] In this embodiment, the processor 401 in the electronic device will load the executable code corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application program stored in the memory 402, thereby achieving:
[0120] receiving a first read request;
[0121] Acquire information to be evaluated corresponding to the first read request and used for performing preloading evaluation;
[0122] If the information to be evaluated meets the preloading condition, determining a first preloading interval according to a first data interval in which the data requested to be read by the first read request is located, and the amount of preloaded data in the first preloading interval is greater than the target data amount;
[0123] The preloaded data in the disk is loaded into the memory.
[0124] In an optional embodiment, when the processor 401 executes the acquisition of the information to be evaluated corresponding to the first read request for performing the preloading evaluation, it can execute: acquiring a first data interval where the data requested to be read by the first read request is located; acquiring a second data interval where the data requested to be read by the second read request received before receiving the first read request is located; acquiring a third data interval where the data requested to be read by the third read request received before receiving the second read request is located; and determining the first data interval, the second data interval and the third data interval as the information to be evaluated.
[0125] In an optional embodiment, before the processor 401 executes the determination of the first data interval, the second data interval, the third data interval and the first virtual pre-read interval as the information to be evaluated, it can also execute: if the current scene is a preloading scene, then obtain the second preloading interval corresponding to the preloading scene, and determine the first virtual pre-read interval based on the first data interval, and the preloading scene is a scene in which the data in the second preloading interval on the disk is loaded into the memory; when the processor 401 executes the determination of the first data interval, the second data interval and the third data interval as the information to be evaluated, it can execute: determine the first data interval, the second data interval, the third data interval, the second preloading interval and the first virtual pre-read interval as the information to be evaluated.
[0126] In an optional embodiment, the processor 401 may further execute: if the first virtual pre-read interval is not in the second pre-load interval, the first data interval and the second data interval meet the conditions for triggering pre-reading, and the second data interval and the third data interval meet the conditions for triggering pre-reading, then determining that the information to be evaluated meets the pre-load condition.
[0127] In an optional embodiment, the processor 401 may also execute: if the first virtual pre-read interval is in the second pre-loading interval, and the first data interval and the second data interval do not meet the conditions for triggering pre-reading, then stop loading the data in the disk that is in the second pre-loading interval and has not been loaded into the memory.
[0128] In an optional embodiment, when the processor 401 executes the determination of the first preloading interval according to the first data interval where the data requested to be read by the first read request is located, it can execute: determining a first virtual pre-reading interval according to the first data interval; determining the first preloading interval according to the first virtual pre-reading interval, the starting position of the first preloading interval is the same as the starting position of the first virtual pre-reading interval, and the size of the first preloading interval is larger than the size of the first virtual pre-reading interval.
[0129] In an optional embodiment, when the processor 401 executes the step of loading the preloaded data in the disk into the memory, the processor 401 may execute: dividing the first preload interval into a plurality of continuous sub-preload intervals; sequentially loading the preloaded data in each of the sub-preload intervals in the disk into the memory; in the process of loading the preloaded data in the corresponding sub-preload interval, if a fourth read request is received, processing the fourth read request after the preloaded data in the corresponding sub-preload interval is loaded.
[0130] In the above embodiments, the description of each embodiment has its own focus. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the data loading method above, and will not be repeated here.
[0131] The data loading device provided in the embodiment of the present application belongs to the same concept as the data loading method in the above embodiment. Any method provided in the data loading method embodiment can be run on the data loading device. The specific implementation process is detailed in the data loading method embodiment, which will not be repeated here.
[0132] It should be noted that, for the data loading method of the embodiment of the present application, a person of ordinary skill in the art can understand that all or part of the process of implementing the data loading method of the embodiment of the present application can be completed by controlling the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, such as a memory, and executed by at least one processor, and the execution process may include the process of the embodiment of the data loading method. Among them, the computer-readable storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.
[0133] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
[0134] For the data loading device of the embodiment of the present application, each functional module can be integrated into a processing chip, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk.
[0135] The above is a detailed introduction to a data loading method, device, storage medium and electronic device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A data loading method, characterized in that: include: receiving a first read request; Acquire information to be evaluated corresponding to the first read request and used for performing preloading evaluation; If the information to be evaluated meets the preloading condition, determining a first preloading interval according to a first data interval in which the data requested to be read by the first read request is located, and the amount of preloaded data in the first preloading interval is greater than the target data amount; The preloaded data in the disk is loaded into the memory.
2. The data loading method according to claim 1, characterized in that: The obtaining information to be evaluated corresponding to the first read request and used for performing preloading evaluation includes: Obtaining a first data interval where the data requested to be read by the first read request is located; Acquire a second data interval where data requested to be read by a second read request received before receiving the first read request is located; Acquire a third data interval where data requested to be read by a third read request received before receiving the second read request is located; The first data interval, the second data interval, and the third data interval are determined as the information to be evaluated.
3. The data loading method according to claim 2, characterized in that: Before determining the first data interval, the second data interval, the third data interval and the first virtual pre-read interval as the information to be evaluated, the method further includes: If the current scene is a preload scene, a second preload interval corresponding to the preload scene is obtained, and a first virtual pre-read interval is determined according to the first data interval, wherein the preload scene is a scene in which data in the second preload interval in the disk is loaded into the memory; The determining the first data interval, the second data interval, and the third data interval as the information to be evaluated includes: The first data interval, the second data interval, the third data interval, the second preloading interval, and the first virtual pre-reading interval are determined as the information to be evaluated.
4. The data loading method according to claim 3, characterized in that: The method further comprises: If the first virtual pre-read interval is not in the second pre-load interval, the first data interval and the second data interval meet the pre-reading triggering condition, and the second data interval and the third data interval meet the pre-reading triggering condition, it is determined that the information to be evaluated meets the pre-load condition.
5. The data loading method according to claim 3, characterized in that: The method further comprises: If the first virtual pre-read interval is in the second pre-load interval, and the first data interval and the second data interval do not satisfy the condition for triggering pre-reading, the loading of the data in the second pre-load interval in the disk that has not been loaded into the memory is stopped.
6. The data loading method according to claim 1, characterized in that: The determining a first preloading interval according to a first data interval where the data requested to be read by the first read request is located includes: Determine a first virtual pre-reading interval according to the first data interval; The first preloading interval is determined according to the first virtual prereading interval, a starting position of the first preloading interval is the same as a starting position of the first virtual prereading interval, and a size of the first preloading interval is larger than a size of the first virtual prereading interval.
7. The data loading method according to claim 1, characterized in that: The step of loading the preloaded data in the disk into the memory includes: Dividing the first preloading interval into a plurality of continuous sub-preloading intervals; sequentially loading the preloaded data in each sub-preload interval in the disk into the memory; During the process of loading the preload data in the corresponding sub-preload interval, if a fourth read request is received, the fourth read request is processed after the preload data in the corresponding sub-preload interval is loaded completely.
8. A data loading device, characterized in that: include: A request receiving module, configured to receive a first read request; An information acquisition module, used to acquire information to be evaluated corresponding to the first read request and used for performing preloading evaluation; an interval determination module, configured to determine a first preloading interval according to a first data interval in which the data requested to be read by the first read request is located if the information to be evaluated meets the preloading condition, and the amount of preloaded data in the first preloading interval is greater than a target data amount; The data loading module is used to load the preloaded data in the disk into the memory.
9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the data loading method according to any one of claims 1 to 7.
10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the data loading method according to any one of claims 1 to 7 by calling the computer program stored in the memory.