Data processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to the field of computer technology, and provides a data processing method, device, electronic device and storage medium, the method comprising: obtaining hard disk load information, determining an IO mode based on the hard disk load information, if the IO mode is the first IO mode, sorting each target request according to the request address size of each target request, and storing the data of each target request in a temporary storage area based on the sorting result, in response to the data stored in the temporary storage area meeting the preset request condition, accessing the hard disk based on each target request stored in the temporary storage area, and obtaining the target data corresponding to each target request. By adopting this method, by monitoring the hard disk load information, determining an adapted IO mode based on the hard disk load information, sorting the target requests according to the request address size in the first IO mode, and then accessing the hard disk in sequence, the seek time of the target request is reduced, the access delay is reduced, and the system performance is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] At present, when the operating system reads data from the physical hard disk, it is generally based on the pre-read mechanism. The data of each adjacent block on the hard disk is read in the pre-read order, and the read data is stored in the cache so that the data can be read from the cache during subsequent access, thereby improving the data access speed. However, for random read access requests, due to the random nature of the request, the hit rate of using the pre-read mechanism for data reading is low, resulting in an increase in the access delay of random read access requests. In addition, due to the limited cache space, frequent data reading operations can easily lead to excessive system load, reducing system performance. Summary of the invention

[0003] The present disclosure provides a data processing method, device, electronic device and storage medium.

[0004] According to a first aspect of the present disclosure, a data processing method is provided, the method comprising:

[0005] Get hard disk load information;

[0006] Determine an IO mode based on the hard disk load information;

[0007] If the IO mode is the first IO mode, sorting each of the target requests according to the request address size of each of the target requests, and storing the data of each of the target requests into a temporary storage area based on the sorting result;

[0008] In response to the data stored in the temporary storage area satisfying a preset request condition, the hard disk is accessed based on each of the target requests stored in the temporary storage area to obtain target data corresponding to each of the target requests.

[0009] In one embodiment, the method further comprises:

[0010] If the IO mode is a second IO mode, each of the target requests is stored in the temporary storage area according to the current request order of each of the target requests, and the second IO mode is different from the first IO mode.

[0011] In one possible implementation, determining the IO mode based on the hard disk load information includes:

[0012] When the current IO mode is the second IO mode, if the hard disk load information indicates that the hard disk is in a high load state, adjusting the IO mode to the first IO mode;

[0013] In the case where the current IO mode is the first IO mode, if the hard disk load information indicates that the hard disk is in a low load state, the IO mode is adjusted to the second IO mode.

[0014] In one possible implementation, the hard disk load information includes hard disk input / output performance parameters and hard disk access delay, and the method further includes:

[0015] If the value of the hard disk input / output performance parameter is greater than a first preset parameter threshold value within a continuous preset number of cycles, and the hard disk access delay is higher than a first preset delay threshold value, it is determined that the hard disk is in a high load state;

[0016] If the value of the hard disk input and output performance parameter in the current cycle is less than the second preset parameter threshold, or the hard disk access delay is lower than the second preset delay threshold, it is determined that the hard disk is in a low load state; wherein the first preset delay threshold is greater than the second preset delay threshold, and the first preset parameter threshold is greater than the second preset parameter threshold.

[0017] In one possible implementation manner, sorting each of the target requests according to the request address size of each of the target requests, and storing the data of each of the target requests into a temporary storage area based on the sorting result, includes:

[0018] Compare the size between the target request and the request addresses of each request stored in the temporary storage area;

[0019] Determine the storage location corresponding to the target request in the temporary storage area according to the address in ascending order;

[0020] The data of the target request is stored in the corresponding storage location in the temporary storage area.

[0021] In one embodiment, the method further comprises:

[0022] If the number of target requests stored in the temporary storage area is greater than a preset number threshold, or the IO buffer time is greater than a preset time threshold, it is determined that the data stored in the temporary storage area meets the preset request condition.

[0023] In one possible implementation, the temporary storage area includes a memory buffer or a preset cache.

[0024] According to a second aspect of the present disclosure, a data processing device is provided, the device comprising:

[0025] A load information acquisition module is used to obtain hard disk load information;

[0026] An IO mode determination module, used to determine the IO mode based on the hard disk load information;

[0027] A sorting module, configured to sort the target requests according to the request address size of each target request if the IO mode is the first IO mode, and store the data of each target request into a temporary storage area based on the sorting result;

[0028] The response module is used for accessing the hard disk based on each of the target requests stored in the temporary storage area in response to the data stored in the temporary storage area satisfying a preset request condition, and obtaining target data corresponding to each of the target requests.

[0029] In one possible implementation, the sorting module is further configured to store each of the target requests into the temporary storage area according to a current request order of each of the target requests if the IO mode is a second IO mode, and the second IO mode is different from the first IO mode.

[0030] In one possible implementation, the IO mode determination module is specifically used to, when the current IO mode is the second IO mode, if the hard disk load information indicates that the hard disk is in a high load state, adjust the IO mode to the first IO mode; when the current IO mode is the first IO mode, if the hard disk load information indicates that the hard disk is in a low load state, adjust the IO mode to the second IO mode.

[0031] In one possible implementation, the hard disk load information includes hard disk input and output performance parameters and hard disk access delay, and the IO mode determination module is specifically used to determine that the hard disk is in a high-load state if the value of the hard disk input and output performance parameter is greater than a first preset parameter threshold value within a continuous preset number of cycles, and the hard disk access delay is higher than the first preset delay threshold; if the value of the hard disk input and output performance parameter in the current cycle is less than a second preset parameter threshold value, or the hard disk access delay is lower than the second preset delay threshold, determine that the hard disk is in a low-load state; wherein the first preset delay threshold is greater than the second preset delay threshold, and the first preset parameter threshold is greater than the second preset parameter threshold.

[0032] In one possible implementation, the sorting module is specifically used to compare the size between the target request and the request address of each request stored in the temporary storage area; determine the storage position corresponding to the target request in the temporary storage area in order of address from small to large; and store the data of the target request into the corresponding storage position in the temporary storage area.

[0033] In one possible implementation, the response module is further used to determine that the data stored in the temporary storage area meets the preset request condition if the number of target requests stored in the temporary storage area is greater than a preset number threshold, or the IO buffer time is greater than a preset time threshold.

[0034] In one possible implementation, the temporary storage area includes a memory buffer or a preset cache.

[0035] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0036] at least one processor; and

[0037] a memory communicatively coupled to the at least one processor;

[0038] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the present disclosure.

[0039] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0040] The data processing method provided by the embodiment of the present disclosure is adopted to obtain the hard disk load information, determine the IO mode based on the hard disk load information, and if the IO mode is the first IO mode, sort the target requests according to the request address size of each target request, and store the data of each target request in a temporary storage area based on the sorting result, and in response to the data stored in the temporary storage area meeting the preset request condition, access the hard disk based on each target request stored in the temporary storage area to obtain the target data corresponding to each target request. That is, by monitoring the hard disk load information, determining the appropriate IO mode based on the hard disk load information, sorting the target requests according to the request address size in the first IO mode, and then accessing the hard disk in sequence, the seek time of the target request is reduced, the access delay is reduced, and the system performance is improved.

[0041] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0043] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0044] Figure 1 A schematic diagram of an implementation flow of a data processing method provided in an embodiment of the present application is shown;

[0045] Figure 2 A schematic diagram of a request sorting process provided by an embodiment of the present application is shown;

[0046] Figure 3 A schematic diagram of a request seek provided by an embodiment of the present application is shown;

[0047] Figure 4 Another schematic diagram of request seeking provided by an embodiment of the present application is shown;

[0048] Figure 5 A schematic diagram of request sorting provided by an embodiment of the present application is shown, wherein (a) is a schematic diagram of a first process of request sorting, (b) is a schematic diagram of a second process of request sorting, (c) is a schematic diagram of a third process of request sorting, (d) is a schematic diagram of a fourth process of request sorting, (e) is a schematic diagram of a fifth process of request sorting, (f) is a schematic diagram of a sixth process of request sorting, (g) is a schematic diagram of a seventh process of request sorting, and (h) is a schematic diagram of an eighth process of request sorting;

[0049] Figure 6 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application is shown;

[0050] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0051] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0052] Since the current use of a pre-reading mechanism for data reading will result in excessive system load and reduce system performance. Therefore, in order to improve system performance, the present application provides a data processing method, device, electronic device and storage medium. The data processing method provided by the present application can be applied to the operating system of electronic devices such as mobile phones, computers and tablet computers.

[0053] The technical solution of the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application.

[0054] Figure 1 A schematic diagram of an implementation flow of the data processing method provided in an embodiment of the present application is shown. Figure 1 As shown, the method includes:

[0055] S101, obtaining hard disk load information.

[0056] In the present disclosure, the operating system can monitor the state of the hard disk and obtain the hard disk load information. For example, the operating system can monitor the load state of the hard disk through a task manager or a graphical interface tool.

[0057] In the present disclosure, the hard disk load information includes but is not limited to the hard disk IOPS (Input / Output Operations Per Second), IO access latency, buffered read IO request volume, IO buffering time and IO response time.

[0058] S102: Determine an IO mode based on the hard disk load information.

[0059] In the present disclosure, the IO mode can be set to include a first IO mode and a second IO mode, and the first IO mode is different from the second IO mode. In the second IO mode, the load of the hard disk is not high, the buffer time waiting for the hard disk IO request is short, the delay is low, the request response of the operating system is fast, and the operating system can process the IO requests in sequence according to the request order of the IO requests.

[0060] S103: If the IO mode is the first IO mode, sort the target requests according to the request address sizes of the target requests, and store the data of the target requests into a temporary storage area based on the sorting result.

[0061] In the present disclosure, the target request may be a read IO request. When the load of the hard disk becomes high, the buffering time waiting for the hard disk IO request is long, the delay is high, and the request response of the operating system is slow, resulting in blocking of new read IO requests. In order to improve the processing efficiency of IO requests, a first IO mode may be designed in the present disclosure. In the first IO mode, the operating system may reorder each IO request according to the request address size of the target request instead of the request order of the IO request, and process the IO requests in sequence according to the new sorting results. By reordering each IO request according to the request address size of the target request, the hard disk head seek time when processing the IO request is reduced, thereby reducing the IO request access delay.

[0062] S104, in response to the data stored in the temporary storage area satisfying a preset request condition, accessing the hard disk based on each of the target requests stored in the temporary storage area to obtain target data corresponding to each of the target requests.

[0063] In the present disclosure, the preset request condition is used to determine whether the data stored in the temporary storage area meets the requirements for accessing the hard disk. For example, if the read IO request stored in the temporary storage area meets the requirements for accessing the hard disk, the operating system can access the hard disk through the read IO request to obtain the hard disk access data.

[0064] The data processing method provided by the embodiment of the present disclosure is adopted to obtain the hard disk load information, determine the IO mode based on the hard disk load information, and if the IO mode is the first IO mode, sort the target requests according to the request address size of each target request, and store the data of each target request in a temporary storage area based on the sorting result, and in response to the data stored in the temporary storage area meeting the preset request condition, access the hard disk based on each target request stored in the temporary storage area to obtain the target data corresponding to each target request. That is, by monitoring the hard disk load information, determining the appropriate IO mode based on the hard disk load information, sorting the target requests according to the request address size in the first IO mode, and then accessing the hard disk in sequence, the seek time of the target request is reduced, the access delay is reduced, and the system performance is improved.

[0065] In the present disclosure, the temporary storage area may include a memory buffer or a preset cache. The preset cache may include a memory cache and a CPU cache.

[0066] In a possible implementation, if the IO mode is the second IO mode, each of the target requests is stored in the temporary storage area according to the current request order of each of the target requests, and the second IO mode is different from the first IO mode. In the second IO mode, the load on the hard disk is not high, the buffer time for waiting for the hard disk IO request is short, and the request delay is low. Therefore, the operating system can process the read IO request according to the current request order of the read IO request. For example, the read IO requests are stored in the temporary storage area in sequence according to the request generation order of the read IO requests, and / or, the hard disk is accessed in sequence through the read IO requests in the order in which the read IO requests are stored in the temporary storage area.

[0067] In a possible implementation manner, determining the IO mode based on the hard disk load information may include steps A1-A2:

[0068] Step A1: when the current IO mode is the second IO mode, if the hard disk load information indicates that the hard disk is in a high load state, the IO mode is adjusted to the first IO mode.

[0069] Step A2: when the current IO mode is the first IO mode, if the hard disk load information indicates that the hard disk is in a low load state, the IO mode is adjusted to the second IO mode.

[0070] In the present disclosure, the operating system can periodically monitor and obtain hard disk load information, and evaluate the hard disk load status in each cycle based on the obtained hard disk load information. For example, the hard disk IOPS can be used to evaluate the amount of read IO requests that the hard disk can process per second in each cycle and the average response delay of the read IO requests in each cycle.

[0071] In the present disclosure, a high load threshold and a low load threshold of the hard disk can be set according to application requirements or according to the historical working status of the hard disk, and the high load threshold is greater than the low load threshold. If it is evaluated that the load of the hard disk in the cycle exceeds the high load threshold, it can be determined that the hard disk is in a high load state, and if it is evaluated that the load of the hard disk in the cycle is less than the low load threshold, it can be determined that the hard disk is in a low load state.

[0072] In the present disclosure, the hard disk load information may include hard disk input / output performance parameters and hard disk access delay. The hard disk input / output performance parameters may be hard disk IOPS. The hard disk access delay may include the average access delay of all read IO requests within a cycle. In a possible implementation, the following steps B1-B2 may be used to determine the hard disk load status:

[0073] Step B1: if the value of the hard disk input / output performance parameter is greater than a first preset parameter threshold value within a continuous preset number of cycles, and the hard disk access delay is higher than a first preset delay threshold value, it is determined that the hard disk is in a high load state.

[0074] In the present disclosure, the preset number can be set according to application requirements, for example, the preset number can be set to 2 or 3, etc. The period for monitoring hard disk input and output performance parameters and hard disk access delay can be set to 5 seconds or 10 seconds, etc. The first preset parameter threshold and the first preset delay threshold are both high load thresholds, which can be set according to actual requirements and are not specifically limited here.

[0075] In the present disclosure, if the value of the hard disk input / output performance parameter is greater than the first preset parameter threshold value and the hard disk access delay is higher than the first preset delay threshold value within a continuous preset number of cycles, it indicates that the concurrency of the read IO request initiated asynchronously by the user application is very high. However, a single hard disk can only synchronously seek a read IO request. The high concurrency of read IO requests will cause the remaining read IO requests to be stored in a temporary storage area waiting for seeking, which puts the hard disk in a high-load state. Therefore, in the case where the current IO mode of the operating system is the second IO mode, if the value of the hard disk input / output performance parameter is greater than the first preset parameter threshold value and the hard disk access delay is higher than the first preset delay threshold value within a continuous preset number of cycles, the IO mode needs to be switched to the first IO mode. In the first IO mode, by sorting each read IO request according to the request address size, the seek time of the read IO request is reduced, thereby reducing the access delay of the read IO request, improving the processing speed of the operating system for the read IO request, and then reducing the hard disk load.

[0076] Step B2, if the value of the hard disk input and output performance parameter in the current cycle is less than the second preset parameter threshold, or the hard disk access delay is lower than the second preset delay threshold, it is determined that the hard disk is in a low load state; wherein the first preset delay threshold is greater than the second preset delay threshold, and the first preset parameter threshold is greater than the second preset parameter threshold.

[0077] In the present disclosure, the second preset parameter threshold and the second preset delay threshold are both low load thresholds and can be set according to actual needs, and are not specifically limited here.

[0078] In the present disclosure, if the value of the hard disk input / output performance parameter in the current cycle is less than the second preset parameter threshold, or the hard disk access delay is lower than the second preset delay threshold, it indicates that the concurrent amount of read IO requests initiated asynchronously by the user application is small, the number of read IO requests that need to be stored in the temporary storage area waiting for seek is reduced and / or the access delay of the read IO request is very low, then the hard disk is in a low load state. Therefore, in the case where the current IO mode of the operating system is the first IO mode, if the value of the hard disk input / output performance parameter in the current cycle is less than the second preset parameter threshold or the hard disk access delay is lower than the second preset delay threshold, the IO mode needs to be switched to the second IO mode, in which the hard disk can support the operating system to process the read IO requests in the order in which the read IO requests are initiated.

[0079] In the present disclosure, the execution order of step B1 and step B2 is not limited.

[0080] In one possible implementation, Figure 2 A schematic diagram of a request sorting process provided by an embodiment of the present application is shown. Figure 2As shown, the sorting of the target requests according to the request address size of each target request, and storing the data of each target request into a temporary storage area based on the sorting result, may include:

[0081] S201, comparing the sizes of the target request and the request addresses of the requests stored in the temporary storage area.

[0082] S202: Determine the storage location corresponding to the target request in the temporary storage area according to the address in ascending order.

[0083] S203: Store the target requested data into the corresponding storage location in the temporary storage area.

[0084] In the present disclosure, taking the target request as a read IO request as an example, the read IO request is stored in the corresponding storage location in the temporary storage area in order of address from small to large, so that the address distance between the read IO request and other adjacent read IO requests in the temporary storage area can be shortened, so that when the operating system accesses the hard disk in sequence according to the read IO requests stored in the temporary storage area, the seek time of the read IO request is shortened, the access delay of the read IO request is reduced, and the processing speed of the operating system for the read IO request is improved.

[0085] For example, Figure 3 FIG. 2 shows a schematic diagram of a request seek provided by an embodiment of the present application. Figure 3 As shown, the sequence of read IO requests is IO1, IO2, IO3, IO4, IO5, IO6, IO7, IO8. The PBA (Logical Block Addressing) addresses corresponding to IO1, IO2, IO3, IO4, IO5, IO6, IO7, and IO8 are PBA1, PBA2, PBA3, PBA4, PBA5, PBA6, PBA7, and PBA8, respectively. PBA1-PBA8 are on the same track, that is, IO1, IO2, IO3, IO4, IO5, IO6, IO7, and IO8 are read IO requests on the same track. Among them, PBA7<PBA4<PBA3<PBA1<PBA8<PBA2<PBA6<PBA5. If the IO mode of the operating system is the first IO mode, sorting each request according to the PBA address size can obtain the sorting result: IO7, IO4, IO3, IO1, IO8, IO2, IO6, and IO5. As Figure 3 As shown, the requests can be stored in the temporary storage area according to the sorting results of each read IO request. When the data stored in the temporary storage area meets the preset request conditions, the hard disk can be accessed according to the order of each read IO request stored in the temporary storage area. For example, Figure 31, 2, 3, 4, 5, 6, 7 and 8 are the sequence numbers corresponding to IO7, IO4, IO3, IO1, IO8, IO2, IO6 and IO5 respectively. Figure 3 As shown, the hard disk can be accessed in the order of IO7, IO4, IO3, IO1, IO8, IO2, IO6, and IO5. Figure 3 As shown, when IO7 initiates a request, the hard disk rotates clockwise until PBA7 rotates to the head, and the data of PBA7 is read, and the IO7 request is completed; after the IO7 request is completed, IO4 initiates a request, and the hard disk can continue to rotate clockwise until PBA4 rotates to the head, and the data of PBA4 is read, and the IO4 request is completed. Similarly, IO1, IO8, IO2, IO6 and IO7 initiate requests in sequence, and the disk rotates clockwise for one circle to read the data of PBA1-PBA8. Compared with accessing the hard disk according to the request sequence of IO1, IO2, IO3, IO4, IO5, IO6, IO7, IO8 in the read IO request sequence, sorting the requests based on the address size and then accessing according to the sorting result can make the disk only rotate one circle to read the data of PBA1-PBA8, greatly shortening the request seek time and reducing the access delay.

[0086] Let me give you another example. Figure 4 Another schematic diagram of request seeking provided by an embodiment of the present application is shown. Figure 4 As shown, the sequence of read IO requests is IO1, IO2, IO3. The Track addresses corresponding to IO1, IO2, IO3 are Track1, Track2, Track3 respectively. Track1, Track2 and Track3 are disk track addresses. Track1, Track2 and Track3 are on different tracks, that is, IO1, IO2 and IO3 are read IO requests for different tracks. Among them, the sequence of read IO requests is IO1, IO2, IO3, and the corresponding PBA addresses are PBA1, PBA2, PBA3 respectively, and PBA3<PBA1<PBA2. If the IO mode of the operating system is the first IO mode, sorting each request according to the PBA address size can obtain the sorting result: IO3, IO1, IO2. Figure 4 1, 2 and 3 are the sorting numbers corresponding to IO3, IO1 and IO2 respectively. Figure 4As shown, the requests can be stored in the temporary storage area according to the sorting results of each read IO request. When the data stored in the temporary storage area meets the preset request conditions, the hard disk can be accessed in the order of each read IO request stored in the temporary storage area. For example, the hard disk can be accessed in the order of IO3, IO1, and IO2. If the head of the disk is currently on Track 2, the read IO request accesses the hard disk in the order of small to large addresses: when IO3 initiates a request, the head of the hard disk moves to Track3 and reads the data of PBA3, then the IO3 request is completed; after the IO3 request is completed, IO1 initiates a request, the head of the hard disk continues to move to Track1 and reads the data of PBA1, then the IO1 request is completed; after the IO1 request is completed, IO2 initiates a request, the head of the hard disk continues to move to Track2 and reads the data of PBA2, then the IO2 request is completed. Compared with accessing the hard disk in the original request order of IO1, IO2 and IO3, when accessing the hard disk in the order after reordering each read IO request according to the first IO mode in the present disclosure, the hard disk only needs to move 4 tracks to complete the access of IO1, IO2 and IO3, and read the data of PBA1-PBA3, which greatly shortens the seek time, reduces the access delay, and improves the access efficiency.

[0087] In a possible implementation, read IO requests can be sorted in ascending order by constructing tree nodes as follows. When the IO mode of the operating system is the first IO mode, for a new random read IO request, the read IO requests of the request tree can be traversed. If the random IO request is a local sequential IO request sequence, read requests can be initiated to the hard disk in ascending order of the PBA address of the read IO request, and the disk of the hard disk starts seeking. When the seek of the read IO request with the smallest PBA address is completed, the next IO request is on the same track or the nearest track, so that the seek of the read IO request can be completed quickly. Figure 5 A schematic diagram of request sorting provided by an embodiment of the present application is shown, wherein (a) is a schematic diagram of the first process of request sorting, (b) is a schematic diagram of the second process of request sorting, (c) is a schematic diagram of the third process of request sorting, (d) is a schematic diagram of the fourth process of request sorting, (e) is a schematic diagram of the fifth process of request sorting, (f) is a schematic diagram of the sixth process of request sorting, (g) is a schematic diagram of the seventh process of request sorting, and (h) is a schematic diagram of the eighth process of request sorting. Figure 5 As shown in the figure, the sequence of read IO requests is IO1, IO2, IO3, IO4, IO5, IO6. The PBA addresses corresponding to IO1, IO2, IO3, IO4, IO5, IO6 are PBA1, PBA2, PBA3, PBA4, PBA5, PBA6. Among them, PBA4<PBA3<PBA1<PBA2<PBA6<PBA5. Figure 5 As shown in (a), based on the order of read IO requests, IO1 can be used as the root node of the tree. When a new read IO request IO2 is generated, the address sizes of IO2 and IO1 are compared. Since PBA2>PBA1, see Figure 5 In (b), IO2 can be regarded as the right child of IO1. The PBA address of the read IO request of the right child is greater than the PBA address of the read IO request of the root node, and the PBA address of the read IO request of the left child is less than the PBA address of the read IO request of the root node. When a new read IO request IO3 is generated, the address size of IO2 is compared with that of IO1. Since PBA3 is less than PBA1, see Figure 5 In (c), IO3 can be regarded as the left child node of IO1. When a new read IO request IO3 is generated, IO4 is compared with the address size of each read IO request. Since PBA4 is smaller than PBA3, see Figure 5 In (d), IO4 can be the left child of IO3. Figure 5 In (e), when IO4 is inserted, IO3 and IO2 are upgraded to parent nodes. When a new read IO request IO5 is generated, the address size of IO5 is compared with that of each read IO request. Since PBA5 is larger than PBA2, see Figure 5 In (f), IO5 can be regarded as the left child of IO2. When a new read IO request IO6 is generated, the address size of IO6 is compared with that of each read IO request. Since PBA6 is smaller than PBA5, see Figure 5 In (g), IO6 can be used as the left child node of IO5. If IO6 is used as the current node, the grandfather node IO2 is rotated left, and the tree nodes are sorted according to the address size of the read IO request, we can get Figure 5 The tree node shown in (h) is composed of Figure 5 (h) in the above example can generate the order of read IO requests: IO4->IO3->IO1->IO2->IO6->IO5. The operating system can access the hard disk in the order of IO4->IO3->IO1->IO2->IO6->IO5. Figure 5 As shown, NUL represents a null node.

[0088] In the present disclosure, if the number of target requests stored in the temporary storage area is greater than a preset number threshold, or the IO buffer time is greater than a preset time threshold, it is determined that the data stored in the temporary storage area meets the preset request condition. In the present disclosure, both the preset number threshold and the preset time threshold can be set according to actual application requirements. For example, the preset time threshold can be set to 2 seconds or 3 seconds, and the preset number threshold can be set to 100 or 200. If the number of target requests stored in the temporary storage area is greater than the preset number threshold or the IO buffer time is greater than the preset time threshold, the operating system can access the hard disk in sequence according to the order of each read IO request stored in the temporary storage area to obtain the access data of each read IO request.

[0089] The data processing method disclosed in the present invention is adopted to design a first IO mode. When the hard disk is under high load, the read IO requests of the buffer are sorted from small to large according to the address through the first IO mode, and the hard disk is accessed in sequence according to the sorting result, thereby reducing the moving track amount of the hard disk head during track seeking, thereby reducing the hard disk track seeking time, reducing the access delay, and thus improving the RAID card performance. In addition, reducing the moving track amount time of the hard disk head during track seeking also increases the mechanical life of the hard disk, reduces power consumption, and reduces power consumption.

[0090] Based on the same inventive concept, according to the data processing method provided by the above embodiment of the present disclosure, correspondingly, another embodiment of the present disclosure further provides a data processing device, whose structural schematic diagram is shown in FIG. Figure 6 As shown, specifically including:

[0091] The load information acquisition module 601 is used to acquire the hard disk load information;

[0092] An IO mode determination module 602, configured to determine an IO mode based on the hard disk load information;

[0093] A sorting module 603 is used for sorting each of the target requests according to the request address size of each of the target requests if the IO mode is the first IO mode, and storing the data of each of the target requests into a temporary storage area based on the sorting result;

[0094] The response module 604 is used to access the hard disk based on each of the target requests stored in the temporary storage area in response to the data stored in the temporary storage area satisfying a preset request condition, and obtain target data corresponding to each of the target requests.

[0095] The data processing device provided by the embodiment of the present disclosure is used to obtain hard disk load information, determine the IO mode based on the hard disk load information, and if the IO mode is the first IO mode, sort each target request according to the request address size of each target request, and store the data of each target request in a temporary storage area based on the sorting result, and in response to the data stored in the temporary storage area meeting the preset request condition, access the hard disk based on each target request stored in the temporary storage area to obtain the target data corresponding to each target request. That is, by monitoring the hard disk load information, determining the appropriate IO mode based on the hard disk load information, sorting the target requests according to the request address size in the first IO mode, and then accessing the hard disk in sequence, the seek time of the target request is reduced, the access delay is reduced, and the system performance is improved.

[0096] In one embodiment, the sorting module 603 is further used to store each of the target requests into the temporary storage area according to the current request order of each of the target requests if the IO mode is a second IO mode, and the second IO mode is different from the first IO mode.

[0097] In one embodiment, the IO mode determination module 602 is specifically used to adjust the IO mode to the first IO mode when the current IO mode is the second IO mode and the hard disk load information indicates that the hard disk is in a high load state; and to adjust the IO mode to the second IO mode when the current IO mode is the first IO mode and the hard disk load information indicates that the hard disk is in a low load state.

[0098] In one possible implementation, the hard disk load information includes hard disk input and output performance parameters and hard disk access delay, and the IO mode determination module 602 is specifically used to determine that the hard disk is in a high-load state if the value of the hard disk input and output performance parameter is greater than a first preset parameter threshold value within a continuous preset number of cycles, and the hard disk access delay is higher than the first preset delay threshold value; if the value of the hard disk input and output performance parameter in the current cycle is less than a second preset parameter threshold value, or the hard disk access delay is lower than the second preset delay threshold value, determine that the hard disk is in a low-load state; wherein, the first preset delay threshold value is greater than the second preset delay threshold value, and the first preset parameter threshold value is greater than the second preset parameter threshold value.

[0099] In one implementation mode, the sorting module 603 is specifically used to compare the size between the target request and the request address of each request stored in the temporary storage area; determine the storage position corresponding to the target request in the temporary storage area in order of the address from small to large; and store the data of the target request into the corresponding storage position in the temporary storage area.

[0100] In one embodiment, the response module 604 is also used to determine that the data stored in the temporary storage area meets the preset request condition if the number of target requests stored in the temporary storage area is greater than a preset number threshold, or the IO buffer time is greater than a preset time threshold.

[0101] In one possible implementation, the temporary storage area includes a memory buffer or a preset cache.

[0102] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0103] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0104] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0105] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0106] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0107] Various embodiments of the technology described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), integrated systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. In the context of the present disclosure, a machine-readable medium may be a tangible medium. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. A computer system may include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. In the description of the present disclosure, “plurality” means two or more than two, unless otherwise clearly and specifically defined.

[0108] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The method comprises: Get hard disk load information; Determine an IO mode based on the hard disk load information; If the IO mode is the first IO mode, sorting each of the target requests according to the request address size of each of the target requests, and storing the data of each of the target requests in a temporary storage area based on the sorting result; the target request is a read IO request; sorting each of the target requests according to the request address size of each of the target requests includes: when the IO mode is the first IO mode, for a new random read IO request, traversing the read IO requests of the request tree; if the random read IO request is a local sequential IO request sequence, initiating read requests to the hard disk in ascending order of the PBA addresses of the read IO requests, so that the disk of the hard disk performs seek; If the number of target requests stored in the temporary storage area is greater than a preset number threshold, or the IO buffer time is greater than a preset time threshold, it is determined that the data stored in the temporary storage area meets the preset request condition; in response to the data stored in the temporary storage area meeting the preset request condition, the hard disk is accessed based on each of the target requests stored in the temporary storage area to obtain the target data corresponding to each of the target requests.

2. The method according to claim 1, characterized in that The method further comprises: If the IO mode is a second IO mode, each of the target requests is stored in the temporary storage area according to the current request order of each of the target requests, and the second IO mode is different from the first IO mode.

3. The method according to claim 1, characterized in that The determining the IO mode based on the hard disk load information includes: When the current IO mode is the second IO mode, if the hard disk load information indicates that the hard disk is in a high load state, adjusting the IO mode to the first IO mode; In the case where the current IO mode is the first IO mode, if the hard disk load information indicates that the hard disk is in a low load state, the IO mode is adjusted to the second IO mode.

4. The method according to claim 3, characterized in that: The hard disk load information includes hard disk input and output performance parameters and hard disk access delay, and the method further includes: If the value of the hard disk input / output performance parameter is greater than a first preset parameter threshold value within a continuous preset number of cycles, and the hard disk access delay is higher than a first preset delay threshold value, it is determined that the hard disk is in a high load state; If the value of the hard disk input and output performance parameter in the current cycle is less than the second preset parameter threshold, or the hard disk access delay is lower than the second preset delay threshold, it is determined that the hard disk is in a low load state; wherein the first preset delay threshold is greater than the second preset delay threshold, and the first preset parameter threshold is greater than the second preset parameter threshold.

5. The method according to claim 1, characterized in that The step of sorting the target requests according to the request address sizes of the target requests, and storing the data of the target requests into a temporary storage area based on the sorting results, comprises: Compare the size between the target request and the request addresses of each request stored in the temporary storage area; Determine the storage location corresponding to the target request in the temporary storage area according to the address in ascending order; The data of the target request is stored in the corresponding storage location in the temporary storage area.

6. The method according to claim 1, characterized in that The temporary storage area includes a memory buffer or a preset cache.

7. A data processing device, characterized in that: The device comprises: A load information acquisition module is used to obtain hard disk load information; An IO mode determination module, used to determine the IO mode based on the hard disk load information; A sorting module, for sorting each target request according to the request address size of each target request if the IO mode is the first IO mode, and storing the data of each target request in a temporary storage area based on the sorting result; the target request is a read IO request; the sorting of each target request according to the request address size of each target request includes: when the IO mode is the first IO mode, for a new random read IO request, traversing the read IO request of the request tree; if the random read IO request is a local sequential IO request sequence, initiating read requests to the hard disk in ascending order of the PBA address of the read IO request, so that the disk of the hard disk performs seek; A response module is used to determine that the data stored in the temporary storage area meets the preset request condition if the number of target requests stored in the temporary storage area is greater than a preset number threshold, or the IO buffer time is greater than a preset time threshold; in response to the data stored in the temporary storage area meeting the preset request condition, access the hard disk based on each of the target requests stored in the temporary storage area to obtain the target data corresponding to each of the target requests.

8. The device according to claim 7, characterized in that The sorting module is further configured to store each of the target requests into the temporary storage area according to a current request order of each of the target requests if the IO mode is a second IO mode, and the second IO mode is different from the first IO mode.

9. The device according to claim 7, characterized in that The IO mode determination module is specifically used to adjust the IO mode to the first IO mode when the current IO mode is the second IO mode and if the hard disk load information indicates that the hard disk is in a high load state; and to adjust the IO mode to the second IO mode when the current IO mode is the first IO mode and if the hard disk load information indicates that the hard disk is in a low load state.

10. The device according to claim 9, characterized in that The hard disk load information includes hard disk input and output performance parameters and hard disk access delay. The IO mode determination module is specifically used to determine that the hard disk is in a high-load state if the value of the hard disk input and output performance parameter is greater than a first preset parameter threshold value within a continuous preset number of cycles, and the hard disk access delay is higher than the first preset delay threshold value; if the value of the hard disk input and output performance parameter in the current cycle is less than a second preset parameter threshold value, or the hard disk access delay is lower than the second preset delay threshold value, determine that the hard disk is in a low-load state; wherein, the first preset delay threshold value is greater than the second preset delay threshold value, and the first preset parameter threshold value is greater than the second preset parameter threshold value.

11. The device according to claim 7, characterized in that The sorting module is specifically used to compare the size between the target request and the request address of each request stored in the temporary storage area; determine the storage position corresponding to the target request in the temporary storage area in order of address from small to large; and store the data of the target request into the corresponding storage position in the temporary storage area.

12. The device according to claim 7, characterized in that The temporary storage area includes a memory buffer or a preset cache.

13. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the program.

14. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to perform the method according to any one of claims 1 to 6 when executed by a computer processor.

Images