Storage node scheduling method, device, storage node, equipment and readable medium

Abstract

The embodiments of the present application provide a scheduling method, device, storage node, electronic device and computer-readable medium for a storage node, and relate to the field of data storage. The method includes: when the storage node receives a write request and determines that the write request is a data write request for the first storage device in the storage node, obtaining characteristic data of the data write request; if it is determined that the characteristic data meets the configured write request redirection condition, determining whether the operating state of the second storage device in the storage node is suitable for the redirection operation of the data write request; if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, performing a redirection operation on the data write request to write the data carried by the data write request to the second storage device. Through the embodiments of the present application, under the premise of meeting the quality of service, the problem of excessive wear of the first storage device due to excessive amount of write data is avoided, and the utilization rate of the second storage device is also improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data storage, and in particular to a storage node scheduling method, device, storage node, electronic device and computer-readable medium. Background Art

[0002] In the current large-scale data storage system, the hardware configuration, architecture and operating cost of storage nodes are the key issues that enterprises need to consider. At present, people often use heterogeneous (different structures) storage media to design heterogeneous storage nodes, so as to reduce the configuration cost of storage nodes while ensuring service quality. By separating hot and cold data, heterogeneous storage nodes put a small amount of hot data into a small-capacity high-performance cache to improve the read access quality of hot data. At the same time, large-scale warm and cold data are put into cheap and high-capacity storage devices for persistence.

[0003] In existing persistent storage systems, due to the absorption of hot data by the cache layer and content distribution network of the upper business, the characteristic of the underlying storage load is often that write operations are higher than read operations. In a single heterogeneous storage node, the solid-state drive is used as the write cache of the disk, that is, most of the data is first written to the solid-state drive, and the high performance of the solid-state drive is used to reduce the data access delay and ensure the service quality; then the data in the solid-state drive is migrated to the disk for persistence. This hierarchical structure can reduce the persistence cost of the storage node while meeting the service requirements. However, this repeated writing strategy for the solid-state drive will not only cause the solid-state drive to wear out too quickly, but also make the disk idle for a long time. Its bandwidth and rewritable characteristics are not fully utilized, resulting in low utilization of the disk. It can be seen that how to effectively solve the problem of data write balance of storage devices with different read and write performance in the storage node has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present application is to propose a storage node scheduling method, device, storage node, electronic device and computer-readable medium, which are used to solve the technical problem of how to effectively solve the data write balance of storage devices with different read and write performance in the storage node in the prior art.

[0005] According to the first aspect of the embodiment of the present application, a scheduling method for a storage node is provided. The method includes: when the storage node receives a write request and determines that the write request is a data write request for a first storage device in the storage node, acquiring characteristic data of the data write request and determining whether the characteristic data satisfies a configured write request redirection condition; if it is determined that the characteristic data satisfies the write request redirection condition, determining whether the operating state of a second storage device in the storage node is suitable for the redirection operation of the data write request, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, performing a redirection operation on the data write request to write the data carried by the data write request into the second storage device.

[0006] According to the second aspect of the embodiment of the present application, a scheduling device for a storage node is provided. The device includes: a first determination module, which is used to obtain characteristic data of the data write request and determine whether the characteristic data meets the configured write request redirection condition when the storage node receives a write request and determines that the write request is a data write request of a first storage device in the storage node; a second determination module, which is used to determine whether the operating state of the second storage device in the storage node is suitable for the redirection operation of the data write request if it is determined that the characteristic data meets the write request redirection condition, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; a first writing module, which is used to perform a redirection operation on the data write request if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, so as to write the data carried by the data write request into the second storage device.

[0007] According to a third aspect of an embodiment of the present application, a storage node is provided. The storage node includes: a scheduler set in the storage node, the scheduler is specifically used to: when the storage node receives a write request and determines that the write request is a data write request for a solid-state hard disk in the storage node, obtain characteristic data of the data write request, and determine whether the characteristic data meets the configured write request redirection condition; if it is determined that the characteristic data meets the write request redirection condition, determine whether the operating state of the disk in the storage node is suitable for the redirection operation of the data write request; if it is determined that the operating state of the disk is suitable for the redirection operation of the data write request, perform a redirection operation on the data write request to write the data carried by the data write request to the disk.

[0008] According to the fourth aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a computer-readable medium configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the scheduling method for storage nodes as described in the first aspect of the above embodiment.

[0009] According to a fifth aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the scheduling method of the storage node as described in the first aspect of the above embodiment is implemented.

[0010] According to the scheduling scheme of the storage node provided by the embodiment of the present application, when the storage node receives a write request and determines that the write request is a data write request for the first storage device in the storage node, characteristic data of the data write request is obtained, and it is determined whether the characteristic data meets the configured write request redirection condition; if it is determined that the characteristic data meets the write request redirection condition, it is determined whether the operating state of the second storage device in the storage node is suitable for the redirection operation of the data write request, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, a redirection operation is performed on the data write request to write the data carried by the data write request to the second storage device. Compared with other existing methods, when the operating state of the second storage device is suitable for the redirection operation, the data carried by the data write request whose characteristics meet the redirection condition originally sent to the first storage device is written to the second storage device. On the premise of ensuring that the data write request meets the service quality, the problem of excessive wear of the first storage device due to excessive write data volume is effectively avoided, and the utilization rate of the second storage device is also effectively improved, thereby effectively ensuring the data write balance of storage devices with different read and write performances in the storage node. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0012] Figure 1A This is a flowchart of the steps of the scheduling method of the storage node in the first embodiment of the present application;

[0013] Figure 1B A schematic diagram of a scheduling process of a data write request of a first storage device provided according to Embodiment 1 of the present application;

[0014] Figure 2A This is a flowchart of the steps of the scheduling method of the storage node in the second embodiment of the present application;

[0015] Figure 2BA schematic diagram of a scheduling process of a write log request of a first storage device provided according to Embodiment 2 of the present application;

[0016] Figure 2C A schematic diagram of a processing thread processing a write request according to Embodiment 2 of the present application;

[0017] Figure 3A This is a flowchart of the steps of the scheduling method of the storage node in the third embodiment of the present application;

[0018] Figure 3B A schematic diagram of a scheduling process of a data write request of a second storage device provided according to Embodiment 3 of the present application;

[0019] Figure 4 A schematic diagram of a storage node scheduling process provided according to Embodiment 4 of the present application;

[0020] Figure 5 This is a schematic diagram of the structure of a scheduling device for a storage node in Embodiment 5 of the present application;

[0021] Figure 6 This is a schematic diagram of the structure of a scheduling device for a storage node in Embodiment 6 of the present application;

[0022] Figure 7 This is a schematic diagram of the structure of a scheduling device for a storage node in Embodiment 7 of the present application;

[0023] Figure 8 This is a schematic diagram of the structure of an electronic device in Embodiment 8 of the present application;

[0024] Fig. 9 This is the hardware structure of the electronic device in Embodiment 9 of the present application. DETAILED DESCRIPTION

[0025] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only configured to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0026] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] Reference Figure 1A , showing a step flow chart of the storage node scheduling method of embodiment 1 of the present application.

[0028] Specifically, the storage node scheduling method of this embodiment includes the following steps:

[0029] In step S101, when a storage node receives a write request and determines that the write request is a data write request of a first storage device in the storage node, characteristic data of the data write request is acquired.

[0030] In an embodiment of the present application, the storage node may be a heterogeneous storage node, and the heterogeneous storage node may be understood as a node composed of storage media with different structures. The first storage device and the second storage device may both be understood as devices for storing data, and the read and write performance of the first storage device is better than the read and write performance of the second storage device. For example, the first storage device may be a solid-state hard disk, and the second storage device may be a magnetic disk. The characteristic data may include at least one of the following: the size of the data carried by the data write request, the sender of the data write request, and the file type to which the data carried by the data write request belongs. It can be understood that the above description is only exemplary, and the embodiment of the present application does not impose any limitation on this.

[0031] In some optional embodiments, before acquiring the characteristic data of the data write request, the method further includes: if the length of the request queue of the first storage device acquired exceeds the first preset length, marking the data write request as a redirection request, and determining whether the operating state of the second storage device is suitable for the redirection operation of the data write request; if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, performing a redirection operation on the data write request to write the data carried by the data write request to the second storage device. Among them, the first preset length can be set by those skilled in the art according to actual needs, and the embodiment of the present application does not make any limitation to this. Thereby, when the operating state of the second storage device is suitable for the redirection operation, the data carried by the data write request whose request queue of the corresponding first storage device exceeds the first preset length is written to the second storage device, further avoiding the problem of excessive wear of the first storage device due to excessive amount of write data, and further improving the utilization rate of the second storage device, thereby further ensuring the balance of data write of storage devices with different read and write performance in the storage node. It can be understood that the above description is only exemplary, and the embodiment of the present application does not make any limitation to this.

[0032] In a specific example, when a storage node receives an IO request to write to a solid-state hard disk, the IO scheduler obtains the length of the request queue of the destination solid-state hard disk of the current IO request (the solid-state hard disk corresponding to the IO request). If the queue length exceeds the set threshold, the request is marked with a redirection tag. If the operating status of the disk meets the redirection operation of the request, the request with the redirection tag is placed in the disk request queue, and the data carried by the request is placed in the disk write cache area. Otherwise, the redirection tag of the request is revoked, it is placed in the destination solid-state hard disk request queue, and the data carried by the request is placed in the solid-state hard disk write cache area. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0033] In step S102, it is determined whether the characteristic data satisfies the configured write request redirection condition.

[0034] In an embodiment of the present application, the redirection condition can be understood as a condition that needs to be satisfied by a data write request for a first storage device when the data carried by the data write request for the first storage device is stored in a second storage device. For example, when the characteristic data includes at least one of the following: the size of the data carried by the data write request, the sender of the data write request, and the file type to which the data carried by the data write request belongs, the write request redirection condition includes at least one of the following: the size of the data carried by the data write request exceeds a configured threshold, the sender of the data write request is a set sender, and the file type to which the data carried by the data write request belongs is a set file type. It can be understood that the above description is only exemplary, and the embodiment of the present application does not impose any limitation on this.

[0035] In some optional embodiments, before determining that the characteristic data satisfies the write request redirection condition, the method further includes: if it is determined that the size of the data carried by the data write request exceeds a configured threshold, and / or the sender of the data write request is a set sender, and / or the file type to which the data carried by the data write request belongs is a set file type, then determining that the characteristic data satisfies the write request redirection condition. Thus, by the size of the data carried by the data write request, the sender of the data write request, and the file type to which the data carried by the data write request belongs, it is possible to accurately determine whether the characteristic data of the data write request satisfies the configured write request redirection condition. It will be understood that the above description is merely exemplary and the embodiments of the present application do not impose any limitation on this.

[0036] In a specific example, when a storage node receives an IO request to write to a solid-state drive, and the length of the request queue of the target solid-state drive of the current IO request does not exceed the set threshold, the IO scheduler obtains the request characteristics of the IO request and marks the request that meets the redirection condition with a redirection tag. It is understandable that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0037] In some optional embodiments, when the characteristic data includes the size of the data carried by the data write request, the method further includes: collecting log data of the data write request received by the storage node in the historical time period, and performing data analysis on the log data to obtain a static threshold value for comparison with the size of the data carried by the data write request; if it is detected that the number of data write requests of the second storage device in the current time period exceeds a preset number, and the amount of data carried by the data write request of the second storage device in the current time period exceeds a preset amount of data, then increasing the static threshold value according to a preset ratio to obtain a dynamic threshold value for comparison with the size of the data carried by the data write request; if it is detected that the number of data write requests of the second storage device in the current time period does not exceed a preset number, and the amount of data carried by the data write request of the second storage device in the current time period does not exceed a preset amount of data, then reducing the static threshold value according to a preset ratio to obtain a dynamic threshold value for comparison with the size of the data carried by the data write request. Wherein, the preset number and the preset amount of data can be set by those skilled in the art according to actual needs, and the embodiments of the present application do not impose any limitation on this. Thus, when the characteristic data includes the size of the data carried by the data write request, the threshold value for comparison with the size of the data carried by the data write request can be dynamically adjusted according to the density of the data write request of the second storage device. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0038] In a specific example, if the redirection condition satisfied by a data write request is that the size of the data carried by the request exceeds a set threshold, the threshold will change dynamically with the density of the data write request. Specifically, first, a static threshold T is set according to the size of the data carried by the data write request in the offline request record to ensure that the processing time after the data write request with a size of more than 90% of T is redirected to the disk can meet the service requirements. When the IO scheduler is running online, when the data write requests are dense, T will be increased by a certain proportion to reduce the data write requests that may be redirected to reduce the disk pressure; when the data write requests are sparse, T will be reduced by a certain proportion to increase the data write requests that may be redirected to improve the disk utilization. It will be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0039] In step S103, if it is determined that the characteristic data meets the write request redirection condition, it is determined whether the operating state of the second storage device in the storage node is suitable for the redirection operation of the data write request.

[0040] In the embodiment of the present application, the operating state of the second storage device includes at least one of the following: the length of the request queue of the second storage device, the size of the data to be written in the write buffer area of ​​the second storage device, the usage rate of the storage space of the second storage device, and the average time for the second storage device to write data. It can be understood that the above description is only exemplary and the embodiment of the present application does not impose any limitation on this.

[0041] In some optional embodiments, before determining that the operating state of the second storage device is suitable for the redirection operation of the data write request, the method further includes: if the result of the determination includes at least one of the following: the length of the request queue of the second storage device does not exceed the second preset length, the size of the data to be written in the write buffer area of ​​the second storage device does not exceed the preset threshold, the usage rate of the storage space of the second storage device does not exceed the preset usage rate threshold, and the average time for the second storage device to write data does not exceed the first preset time, then determine that the operating state of the second storage device is suitable for the redirection operation of the data write request. Among them, the second preset length, the preset threshold, the preset usage rate threshold, and the first preset time can be set by those skilled in the art according to actual needs, and the embodiment of the present application does not make any limitation on this. Thus, by the length of the request queue of the second storage device, the size of the data to be written in the write buffer area of ​​the second storage device, the usage rate of the storage space of the second storage device, or the average time for the second storage device to write data, it is possible to accurately determine whether the operating state of the second storage device is suitable for the redirection operation of the data write request. It can be understood that the above description is only exemplary, and the embodiment of the present application does not make any limitation on this.

[0042] In a specific example, the IO scheduler monitors the operating status of the disk online. If the operating status of the disk satisfies the redirection operation of the data write request, the data write request with the redirection tag is placed in the disk request queue, and the data carried by the data write request is placed in the disk write cache area. Otherwise, the redirection tag of the data write request is revoked, and it is placed in the request queue of the destination solid-state drive, and the data carried by the data write request is placed in the solid-state drive write cache area. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0043] In step S104, if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, a redirection operation is performed on the data write request to write the data carried by the data write request into the second storage device.

[0044] In some optional embodiments, when performing a redirection operation on the data write request to write the data carried by the data write request into the second storage device, the data write request is stored in the request queue of the second storage device, and the data carried by the data write request is stored in the write cache area of ​​the second storage device; a processing thread is used to poll the request queue of the second storage device to obtain the data write request in the request queue of the second storage device, and according to the size of the data carried by the data write request, the data carried by the data write request is obtained from the write cache area of ​​the second storage device, and then the data carried by the data write request is written into the file of the second storage device. Thus, by using a processing thread to perform a redirection operation on the data write request, the data carried by the data write request can be accurately written into the second storage device. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0045] In some optional embodiments, after writing the data carried by the data write request into the second storage device, the method further includes: using the processing thread to record metadata of the data write request. Thus, by recording the metadata of the data write request, it is possible to conveniently and accurately read and write access to the data carried by the data write request. It is to be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0046] In a specific example, the metadata of the data write request includes at least one of the following: the size of the data carried by the data write request, the offset of the data carried by the data write request in the file to which it belongs, and the physical address of the second storage device. When using the processing thread to record the metadata of the data write request, the size of the data carried by the data write request, the offset of the data carried by the data write request in the file to which it belongs, and the physical address of the second storage device are combined into a new triple; the file name to which the data carried by the data write request belongs is used as a key to query the hash table to determine whether the key exists in the hash table; if the key exists in the hash table, the value corresponding to the key is obtained from the hash table, and according to the value, a linked list of the metadata of the file is obtained, and then a new node with the value of the newly created triple is added to the end of the linked list. In this way, by using the hash table and the linked list of the metadata of the file, the metadata of the data write request can be accurately recorded. It can be understood that the above description is only exemplary, and the embodiments of the present application do not make any limitation to this.

[0047] In a specific example, the method further includes: if the key does not exist in the hash table, creating a new linked list, and adding a new node with the value of the newly created triplet at the end of the new linked list; inserting the head pointer of the new linked list into the hash table as a value, and inserting the file name to which the data carried by the data write request belongs as a key into the hash table. Thus, when the corresponding key does not exist in the hash table, the metadata of the data write request can be recorded by creating a new linked list, so that the metadata of the data write request can be accurately recorded. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0048] In a specific example, a hash table whose key is the file name and whose value is the metadata index of the file is maintained in the memory. The file metadata index points to the metadata of the file. The metadata of the file is maintained using a linked list, and the value of each node in the linked list is a triple consisting of the size of the data carried by the data write request, the offset of the data carried by the data write request in the file to which it belongs, and the physical address of the storage device of the data carried by the data write request. The processing thread polls the request queue and obtains the data write request from the request queue, and then takes out the data from the write buffer and writes it to the corresponding storage device according to the size of the data carried by the data write request. After the writing is completed, the metadata of the data write request is recorded. Specifically, after the data writing is completed, if the write request is a data write request, the processing thread first forms a triple with the size of the data carried by the data write request, the offset of the data carried by the data write request in the file to which it belongs, and the physical address of the storage device of the data carried by the data write request. Then, the file name of the file to which the data carried in the data write request in the triple belongs is used as the key to query the hash table. If the key does not exist, a new linked list is created, and a new node with the value of the newly created triple is added to the end of the new linked list. Finally, the head pointer of the new linked list is used as the value and the file name of the file to which the data carried in the data write request belongs is inserted into the hash table together with the key. If the key exists, the value corresponding to the key is obtained, and the file metadata linked list is obtained according to the value, and a new node with the value of the newly created triple is appended to the end of the linked list. It can be understood that the above description is only exemplary, and the embodiments of the present application do not make any limitation to this.

[0049] In some optional embodiments, when the number of the second storage devices whose operating state is suitable for the redirection operation of the data write request is multiple, the method further includes: adopting a load balancing principle or a load concentration principle to select the second storage device for writing the data carried by the data write request from the multiple second storage devices. Thus, by adopting the load balancing principle or the load concentration principle, it is possible to accurately select a second storage device suitable for writing the data carried by the data write request from the multiple second storage devices. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0050] In a specific example, the IO scheduler can select the destination storage device for the request from a group of storage devices according to the following two principles. The load balancing principle can be understood as the principle of ensuring that the amount of data on each storage device is balanced. Specifically, the IO scheduler obtains the capacity utilization rate of all second storage devices whose operating status meets the redirection operation of the data write request at the current moment, and then calculates the average capacity utilization rate, and randomly allocates a second storage device to the data write request from the second storage devices whose current capacity utilization rate is less than the average capacity utilization rate and whose remaining capacity is greater than the size of the data carried by the data write request. The load concentration principle can be understood as the principle of preferentially filling the storage space of a storage device. Specifically, the IO scheduler obtains the capacity utilization rate of all second storage devices whose operating status meets the redirection operation of the data write request at the current moment and sorts them from large to small. Under the condition that the remaining capacity of the second storage device is greater than the size of the data carried by the data write request, the second storage device with a large capacity utilization rate is preferentially allocated to the data write request. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0051] In step S105, if it is determined that the characteristic data does not meet the write request redirection condition, or the operating state of the second storage device is not suitable for the redirection operation of the data write request, the data carried by the data write request is written to the corresponding first storage device.

[0052] In a specific example, Figure 1BAs shown, it is assumed that the write request redirection condition is that the size of the data carried by the data write request exceeds the set threshold T, and the condition that the disk's operating status satisfies the data write request redirection operation is that the length of the disk's request queue does not exceed the set threshold L. When the storage node receives an IO request to write to the solid-state hard disk, the specific process steps are as follows: (1) The storage node receives an IO request to write to the solid-state hard disk, and the IO scheduler obtains the length of the request queue of the destination solid-state hard disk S of the current IO request. If the length is greater than the set threshold, go to (4), otherwise go to (2); (2) The IO scheduler obtains the size of the data carried by the data write request and goes to (3); (3) Is the size of the data carried by the data write request greater than the set threshold T? If so, go to (4), otherwise go to (7); (4) The IO scheduler obtains the length of the request queue of all disks and determines the length of the request queue. Whether the length exceeds the set threshold L, if both exceed, go to (7), otherwise go to (5); (5) The IO scheduler obtains the capacity utilization rate of the disk whose current queue length does not exceed the set threshold L at the current moment, and then calculates the average capacity utilization rate, and randomly selects a disk H from the disks whose current capacity utilization rate is less than the average capacity utilization rate and whose remaining capacity is greater than the size of the data carried by the data write request, and go to (6); (6) The IO scheduler inserts the data write request into the request queue of disk H, and go to (8); (7) The IO scheduler inserts the data write request into the request queue of the target solid state drive S, and go to (8); (8) The scheduling ends. It can be understood that the above description is only exemplary, and the embodiments of the present application do not make any limitation to this.

[0053] Through the scheduling method of the storage node provided by the embodiment of the present application, when the storage node receives a write request and determines that the write request is a data write request for the first storage device in the storage node, characteristic data of the data write request is obtained, and it is determined whether the characteristic data meets the configured write request redirection condition; if it is determined that the characteristic data meets the write request redirection condition, it is determined whether the operating state of the second storage device in the storage node is suitable for the redirection operation of the data write request, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, a redirection operation is performed on the data write request to write the data carried by the data write request to the second storage device. Compared with other existing methods, when the operating state of the second storage device is suitable for the redirection operation, the data carried by the data write request whose request characteristics originally sent to the first storage device meet the redirection condition is written to the second storage device. On the premise of ensuring that the data write request meets the service quality, the problem of excessive wear of the first storage device due to excessive write data volume is effectively avoided, and the utilization rate of the second storage device is also effectively improved, thereby effectively ensuring the data write balance of storage devices with different read and write performances in the storage node.

[0054] The scheduling method of the storage node of this embodiment can be executed by any appropriate device with data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, vehicle-mounted devices, entertainment equipment, advertising equipment, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus Rift, Hololens, Gear VR), etc.

[0055] Reference Figure 2A , showing a step flow chart of the storage node scheduling method of the second embodiment of the present application.

[0056] Specifically, the storage node scheduling method of this embodiment includes the following steps:

[0057] In step S201, when the write request received by the storage node is a write log request of the first storage device, the write log request is cached in a log file in a write log cache area.

[0058] In an embodiment of the present application, the storage node may be a heterogeneous storage node, that is, a node composed of storage media of different structures. The write log request may be understood as a request to write the carried log data. It is understood that the above description is only exemplary and the embodiment of the present application does not make any limitation to this.

[0059] In some optional embodiments, after storing the write log request in the log file of the write log buffer area, the method further includes: recording the file name of the log file to which the write log request belongs and the offset of the write log request in the log file to which it belongs. Thus, by recording the file name of the log file to which the write log request belongs and the offset of the write log request in the log file to which it belongs, it is possible to conveniently and accurately read and write the log data carried by the write log request. It is to be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0060] In step S202, when the log write buffer is full, a merge operation is performed on the log write requests in the log files in the log write buffer to obtain a large log write request.

[0061] In an embodiment of the present application, when the remaining capacity of the cache space of the write log cache area is zero, a merge operation can be performed on the write log request in the log file of the write log cache area based on the metadata of the write log request to obtain a large write log request. Wherein, the metadata of the write log request may include at least one of the following: the file name of the log file to which the write log request belongs and the offset of the write log request in the log file to which it belongs. It can be understood that the above description is only exemplary and the embodiment of the present application does not impose any limitation on this.

[0062] In step S203, when it is determined that the operating state of the second storage device is suitable for the redirection operation of the large write log request, a redirection operation is performed on the large write log request to write the log data carried by the large write log request into the second storage device.

[0063] In the embodiment of the present application, the specific implementation of determining whether the operating state of the second storage device is suitable for the redirection operation of the large write log request is similar to the specific implementation of determining whether the operating state of the second storage device is suitable for the redirection operation of the data write request in the above-mentioned embodiment 1, and will not be repeated here. The specific implementation of performing a redirection operation on the large write log request to write the log data carried by the large write log request to the second storage device is similar to the specific implementation of performing a redirection operation on the data write request in the above-mentioned embodiment 1 to write the data carried by the data write request to the second storage device, and will not be repeated here. It can be understood that the above description is only exemplary, and the embodiment of the present application does not impose any limitation on this.

[0064] In a specific example, when the storage node receives a write log request with dense intervals, the same size and a small size sent to the solid-state drive, the IO scheduler puts the write log request into a fixed-size write log buffer, and records the offset of the write log request in the log file to which it belongs and the file name of the log file to which the write log request belongs. When the write log buffer is full, multiple write log requests in the log file of the write log buffer are merged into a large write log request. When the running state of the disk meets the redirection operation of the large write log request, the large write log request is placed in the disk request queue, and the log data in the write log buffer (the log data carried by the large write log request) is placed in the disk write buffer; otherwise, the large write log request is placed in the solid-state drive request queue, and the log data in the write log buffer (the log data carried by the large write log request) is placed in the solid-state drive write buffer. It is understandable that the above description is only exemplary, and the embodiments of the present application do not make any limitation to this.

[0065] In some optional embodiments, the method further includes: when the write request received by the storage node is a write log request of the first storage device, caching the write log request into a log file in a write log cache area; polling the status set for the write log cache area by the assigned cache monitoring thread; and based on the status, performing a persistent storage operation on the log data carried by the write log request in the log file of the write log cache area. Thus, by polling the status set for the write log cache area by the assigned cache monitoring thread, and based on the status set for the write log cache area by the cache monitoring thread, performing a persistent storage operation on the log data carried by the write log request in the log file of the write log cache area, the efficiency of performing a persistent storage operation on the log data carried by the write log request in the log file of the write log cache area can be effectively improved. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0066] In some optional embodiments, before the cache monitoring thread assigned by the polling sets the state of the write log cache area, the method further includes: if the write log cache area is full, and the difference between the current time point and the time point when the write log cache area caches the first write log request is less than the second preset duration, then the cache monitoring thread assigned to the write log cache area is used to update the state of the write log cache area to the ready state; if the difference between the current time point and the time point when the write log cache area caches the first write log request is greater than or equal to the second preset duration, then the cache monitoring thread assigned to the write log cache area is used to update the state of the write log cache area to the dump state. Among them, the second preset duration can be set by those skilled in the art according to actual needs, and the embodiment of the present application does not make any limitation to this. Thus, through the remaining capacity of the write log cache area, the current time point, the time point when the write log cache area caches the first write log request, and the second preset duration, the cache monitoring thread of the write log cache area can accurately update the state of the write log cache area. It can be understood that the above description is only exemplary, and the embodiment of the present application does not make any limitation to this.

[0067] In a specific example, the IO scheduler sets a time threshold for the log data in the write log buffer, and assigns a cache monitoring thread to monitor the state of the write log buffer. The cache monitoring thread sets a mark to represent the current state of the write log buffer, which is visible to the IO scheduler and the initial state is the waiting state. When the first write log request is written to the write log buffer, the cache monitoring thread records the time when the request arrives as the starting time point. If the write log buffer is full and the difference between the current time and the starting time point is less than the set time threshold, the cache monitoring thread updates the state of the write log buffer to the ready state. If the difference between the current time and the starting time point is greater than or equal to the set time threshold, regardless of whether the write log buffer is full, the cache monitoring thread will update the state of the write log buffer to the dump state. The purpose of setting the time threshold is to aggregate log data as much as possible during this period and write in large blocks. If the time threshold is exceeded, the log data in the write log buffer must be persisted even if not enough log data is aggregated, so as to ensure the reliability of the log data. In addition, log write requests have a certain timeliness, and the result of the request can only be returned after the log data is persistently stored, and the failure of the log write request cannot be caused by the long aggregation time of the log data in the log write buffer. It is understandable that the above description is only exemplary, and the embodiments of the present application do not make any limitation to this.

[0068] In a specific example, when performing a persistent storage operation on the log data carried by the write log request in the log file of the write log cache area based on the state, when the state is the ready state, a merge operation is performed on the write log request in the log file of the write log cache area to obtain a large write log request; when it is determined that the operating state of the second storage device is suitable for the redirection operation of the large write log request, a redirection operation is performed on the large write log request to write the log data carried by the large write log request into the second storage device. Thus, when the operating state of the second storage device is suitable for the redirection operation, the log data carried by the write log request originally sent to the first storage device is written to the second storage device, and under the premise of ensuring that the write log request meets the service quality, the problem of excessive wear of the first storage device due to excessive amount of write data is further avoided, and the utilization rate of the second storage device is further improved, thereby further ensuring the balance of data writes of storage devices with different read and write performances in the storage node. It can be understood that the above description is only exemplary, and the embodiment of the present application does not make any limitation to this.

[0069] In a specific example, when there are multiple second storage devices whose operating state is suitable for the redirection operation of the large write log request, the method further includes: adopting the load balancing principle or the load concentration principle to select the second storage device for writing the log data carried by the large write log request from the multiple second storage devices. Thus, by adopting the load balancing principle or the load concentration principle, it is possible to accurately select the second storage device suitable for writing the log data carried by the write log request from the multiple second storage devices. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0070] In a specific example, when a persistent storage operation is performed on the log data carried by the write log request in the log file of the write log cache area based on the state, when the state is the dump state, a merge operation is performed on the write log request in the log file of the write log cache area to obtain a large write log request; and the load balancing principle or the load concentration principle is adopted to select the first storage device for writing the log data carried by the large write log request from multiple first storage devices. Thus, by adopting the load balancing principle or the load concentration principle, a first storage device suitable for writing the log data carried by the write log request can be accurately selected from multiple first storage devices. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0071] In a specific example, the IO scheduler polls the cache monitoring thread for the state set for the write log buffer area, and when the polled state is the ready state, multiple write log requests in the write log buffer area are merged into a large write log request. When the disk operation state meets the redirection operation of the large write log request, the request is placed in the disk request queue, and the log data in the write log buffer area (the log data carried by the large write log request) is placed in the disk write buffer area; otherwise, the request is placed in the solid state drive request queue, and the log data in the write log buffer area (the log data carried by the large write log request) is placed in the solid state drive write buffer area. When the polled state is the dump state, the IO scheduler merges all the write log requests in the write log buffer area into a large write log request, puts the large write log request into the request queue of the solid state drive, and puts the log data in the write log buffer area (the log data carried by the large write log request) into the solid state drive write buffer area. The IO scheduler finally updates the state of the write log buffer area to be a waiting state. It is understandable that the above description is only exemplary, and the present application embodiment does not make any limitation to this.

[0072] In a specific example, Figure 2BAs shown, it is assumed that the running state of the disk satisfies the condition for redirecting the write log request if the length of the disk request queue does not exceed the set threshold L. The specific process steps when the storage node receives a write log request sent to the SSD are as follows: (1) The storage node receives the write log request from the SSD, and goes to (2); (2) The IO scheduler caches the log data carried by the request to the write log cache. If the request is the first write log request in the write log cache, the cache monitoring thread records the time when the request arrives as the start time, and goes to (3); (3) Determine whether the difference between the current time point and the start time point is less than the set time threshold. If it is less, go to (4); if it is greater than or equal to, the cache monitoring thread will update the state of the write log cache to the dump state, and go to (8); (4) If the write log cache is full, the cache monitoring thread updates the state of the write log cache to the ready state, and goes to (5); if it is not full, go to (12); (5) The IO scheduler merges multiple write log requests in the write log cache into a large write log request, obtains the length of the request queue of all disks, and determines whether there is a disk whose queue length exceeds the set threshold L. If all of them exceed, go to (9), otherwise go to (6); (6) The IO scheduler obtains The capacity utilization rate of the disk whose request queue length does not exceed the set threshold L at the current moment is then calculated, and the average capacity utilization rate is then calculated. A disk H' is randomly selected from the disks whose current capacity utilization rate is less than the average capacity utilization rate and whose remaining capacity is greater than the size of the log data carried by the large write log request, and then the process proceeds to (7); (7) The IO scheduler puts the large write log request into the request queue of disk H', and then the process proceeds to (11); (8) The IO scheduler merges multiple write log requests in the write log cache into a large write log request, and then the process proceeds to (9); (9) The IO scheduler obtains the capacity utilization rate of all solid state drives at the current moment, and then the average capacity utilization rate is then calculated. A solid state drive S' is randomly selected from the solid state drives whose current capacity utilization rate is less than the average capacity utilization rate and whose remaining capacity is greater than the size of the log data carried by the merged request, and then the process proceeds to (10); (10) The IO scheduler puts the large write log request into the request queue of solid state drive S', and then the process proceeds to (11); (11) The IO scheduler updates the state of the write log cache to the waiting state, and then the process proceeds to (12); (12) The scheduling ends. It should be understood that the above description is merely exemplary and the embodiments of the present application do not impose any limitation thereto.

[0073] In a specific example, Figure 2CAs shown, the specific process steps of the processing thread processing the write request in the request queue are as follows: (1) The processing thread polls the request queue to obtain the write request, obtains the to-be-written data from the corresponding write buffer area according to the size of the data carried by the write request, and writes it to the destination storage device; (2) After the writing is completed, if the request type is a non-write log request, the size of the data carried by the request, the offset of the data carried by the request in the file to which it belongs, and the physical address of the storage device of the data carried by the request are combined into a triple, and go to (3), otherwise go to (5); (3) The file name of the file to which the data carried by the request in the triple belongs is used as the key to query the hash table. If the key does not exist, a new linked list is created, and a value of the new key is added to the end of the linked list. Create a new node of the triple, and finally insert the head pointer of the linked list as the value and the file name of the file to which the data carried by the request belongs as the key into the hash table, and there is a transition (4); (4) Get the value corresponding to the key, obtain the file metadata linked list according to the value, and append the value to the end of the linked list as a new node of the newly created triple; (5) According to the size of the log data carried by each write log request, the offset of the log data carried by each write log request in the file to which it belongs, and the physical address of the storage device of the log data carried by each write log request, multiple triplets are respectively formed, and steps (3) and (4) are repeated until each small write log request in the large write log request is processed; (6) The request processing ends. It can be understood that the above description is only exemplary, and the embodiments of the present application do not make any limitation to this.

[0074] By using the scheduling method for storage nodes provided by the embodiment of the present application, when the write request received by the storage node is a write log request of the first storage device, the write log request is cached in the log file of the write log cache area; when the write log cache area is full, a merge operation is performed on the write log requests in the log file of the write log cache area to obtain a large write log request; when it is determined that the operating state of the second storage device is suitable for the redirection operation of the large write log request, a redirection operation is performed on the large write log request to write the log data carried by the large write log request to the second storage device. Compared with other existing methods, when the operating state of the second storage device is suitable for the redirection operation, the log data carried by the write log request originally sent to the first storage device is written to the second storage device. Under the premise of ensuring that the write log request meets the service quality, the problem of excessive wear of the first storage device due to excessive amount of write data is further avoided, and the utilization rate of the second storage device is further improved, thereby further ensuring the balance of data writing of storage devices with different read and write performances in the storage node.

[0075] The scheduling method of the storage node of this embodiment can be executed by any appropriate device with data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, vehicle-mounted devices, entertainment equipment, advertising equipment, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus Rift, Hololens, Gear VR), etc.

[0076] Reference Figure 3A , showing a step flow chart of the storage node scheduling method of embodiment 3 of the present application.

[0077] Specifically, the storage node scheduling method of this embodiment includes the following steps:

[0078] In step S301, when the current write request received by the storage node is a data write request for the first storage device and the last write request received by the storage node is a data write request for the second storage device, characteristic data of the current write request is acquired.

[0079] In an embodiment of the present application, the storage node may be a heterogeneous storage node, and the heterogeneous storage node may be understood as a node composed of storage media with different structures. The first storage device and the second storage device may both be understood as devices for storing data, and the read and write performance of the first storage device is better than the read and write performance of the second storage device. For example, the first storage device may be a solid-state hard disk, and the second storage device may be a magnetic disk. The characteristic data may include at least one of the following: the size of the data carried by the data write request, the sender of the data write request, and the file type to which the data carried by the data write request belongs. It can be understood that the above description is only exemplary, and the embodiment of the present application does not impose any limitation on this.

[0080] In step S302, it is determined whether the characteristic data of the current write request meets the configured write request redirection condition.

[0081] In the embodiment of the present application, the redirection condition can be understood as the condition that the current write request of the destination storage device, which is the first storage device, needs to meet when the data carried by the current write request of the destination storage device, which is the first storage device, is stored in the second storage device. For example, when the characteristic data includes at least one of the following: the size of the data carried by the current write request, the sender of the current write request, and the file type to which the data carried by the current write request belongs, the write request redirection condition includes at least one of the following: the size of the data carried by the current write request exceeds the configured threshold, the sender of the current write request is the set sender, and the file type to which the data carried by the current write request belongs is the set file type. It can be understood that the above description is only exemplary, and the embodiment of the present application does not impose any limitation on this.

[0082] In step S303, if it is determined that the characteristic data does not satisfy the write request redirection condition, it is determined whether the size of the to-be-written data in the write cache area of ​​the second storage device corresponding to the previous write request exceeds a configured threshold.

[0083] Since the specific implementation of step S303 is similar to the specific implementation of step S103 above, it will not be described again here.

[0084] In step S304, if it is determined that the size of the data to be written in the write cache area of ​​the second storage device corresponding to the previous write request does not exceed the configured threshold, a redirection operation is performed on the current write request to write the data carried by the current write request to the second storage device corresponding to the previous write request.

[0085] Since the specific implementation of step S304 is similar to the specific implementation of step S104 described above, it will not be repeated here.

[0086] In step S305, if it is determined that the characteristic data meets the write request redirection condition, or the size of the data to be written in the write cache area of ​​the second storage device corresponding to the previous write request exceeds the configured threshold, the data carried by the current write request is written to the corresponding first storage device.

[0087] Since the specific implementation of step S305 is similar to the specific implementation of step S105 described above, it will not be repeated here.

[0088] In a specific example, when a node receives a disk write IO request, the request does not pass through the IO scheduler, but directly enters the destination disk request queue, and the data contained in the request is placed in the disk write cache. When the next write request to the solid-state drive arrives, if the request characteristics do not meet the redirection requirements, the IO scheduler determines whether the length of the data to be written in the current disk write cache exceeds the set threshold. If not, the request is labeled "redirect" and placed in the disk request queue, and the data contained in the request is placed in the disk write cache; otherwise, it is placed in the solid-state drive request queue, and the data contained in the request is placed in the solid-state drive write cache. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0089] In a specific example, Figure 3B As shown, when a storage node receives a data write request sent to a disk, the specific process steps are as follows: (1) When a storage node receives a data write request sent to a disk, the data write request directly enters the request queue of the destination disk (the disk corresponding to the data write request), and the data carried by the data write request is placed in the disk write cache area; (2) If the next data write request is a request to write to a solid-state hard disk, if the characteristics of the request do not meet the redirection conditions, go to (3); (3) The IO scheduler determines whether the size of the data to be written in the write cache area of ​​the current disk exceeds the set threshold, if not, go to (4), otherwise go to (5); (4) The request is marked with a redirection tag and placed in the disk request queue, and the data carried by the request is placed in the disk write cache area; (5) The request is placed in the solid-state hard disk request queue, and the data carried by the request is placed in the solid-state hard disk write cache area. It can be understood that the above description is only exemplary, and the embodiments of the present application do not make any limitation to this.

[0090] Through the scheduling method of the storage node provided by the embodiment of the present application, when the current write request received by the storage node is a data write request for the first storage device, and the previous write request received by the storage node is a data write request for the second storage device, the characteristic data of the current write request is obtained, and it is determined whether the characteristic data of the current write request meets the configured write request redirection condition; if it is determined that the characteristic data does not meet the write request redirection condition, it is determined whether the size of the to-be-written data in the write cache area of ​​the second storage device corresponding to the previous write request exceeds the configured threshold value; if it is determined that the size of the to-be-written data in the write cache area of ​​the second storage device corresponding to the previous write request does not exceed the configured threshold value, the current write request is redirected to another storage device. The write request performs a redirection operation to write the data carried by the current write request to the second storage device corresponding to the previous write request. Compared with other existing methods, when the size of the data to be written in the write cache area of ​​the second storage device corresponding to the previous write request does not exceed the configured threshold, the data carried by the current write request originally sent to the first storage device whose characteristics do not meet the redirection conditions are written to the second storage device corresponding to the previous write request, further avoiding the problem of rapid wear of the first storage device due to excessive amount of write data, and further improving the utilization rate of the second storage device, thereby further ensuring the balance of data writing of storage devices with different read and write performance in the storage node.

[0091] The scheduling method of the storage node of this embodiment can be executed by any appropriate device with data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, vehicle-mounted devices, entertainment equipment, advertising equipment, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus Rift, Hololens, Gear VR), etc.

[0092] Embodiment 4 of the present application provides a storage node, which includes: a scheduler set in the storage node, and the scheduler is specifically used to: when the storage node receives a write request and determines that the write request is a data write request for a solid-state hard disk in the storage node, obtain characteristic data of the data write request and determine whether the characteristic data meets the configured write request redirection condition; if it is determined that the characteristic data meets the write request redirection condition, determine whether the operating state of the disk in the storage node is suitable for the redirection operation of the data write request; if it is determined that the operating state of the disk is suitable for the redirection operation of the data write request, perform a redirection operation on the data write request to write the data carried by the data write request to the disk.

[0093] In a specific example, Figure 4As shown, a single heterogeneous storage node includes a group of solid-state hard disks, a group of disks, an IO scheduler, a group of processing threads, and a memory containing a hash table, a write buffer area, a write log buffer area, a request queue corresponding to each solid-state hard disk, and a request queue corresponding to each disk. The hash table is maintained in the memory and is used to record the file name of the data file to which the data carried by the write request belongs, the size of the data carried by the write request, the offset of the data carried by the write request in the data file to which it belongs, and the physical address of the data written to the storage device carried by the write request, and needs to be regularly persisted to the non-volatile storage device. The write buffer area is a buffer in the memory for temporarily storing the data to be written carried by the write request sent to the solid-state hard disk or disk. The write log buffer area is a buffer in the memory for temporarily storing the log data carried by the write log request sent to the solid-state hard disk. The solid-state hard disk request queue is a queue for storing data write requests sent to the solid-state hard disk, and each solid-state hard disk corresponds to a solid-state hard disk request queue. The disk request queue is a queue for storing data write requests sent to the disk, and each disk corresponds to a disk request queue. The processing thread is used to process the data write request or log write request in the request queue, write the data in the write buffer area to the corresponding storage device, and after the writing is completed, the file name of the file to which the data carried by the write request belongs, the size of the data carried by the write request, the offset of the data carried by the write request in the file to which it belongs, and the physical address of the write device of the data carried by the write request are recorded in the metadata linked list. The IO scheduler, according to the length of the current solid-state hard disk request queue, the operating status of the disk, and the characteristic data of the write request, does not directly put all the solid-state hard disk write requests sent by users into the solid-state hard disk request queue, but puts the corresponding solid-state hard disk write request into the disk request queue when the characteristic data meets the redirection condition and the operating status of the disk meets the redirection operation of the write request. The scheduling process of the storage node is as follows: when an IO request to write to the solid-state hard disk arrives, the IO scheduler obtains the request feature of the IO request, and when the disk status meets the redirection operation of the request, puts the IO request whose request feature meets the redirection condition into the disk request queue, and puts the data carried by the IO request into the disk write buffer area; otherwise, puts the request into the solid-state hard disk request queue, and puts the data carried by the request into the solid-state hard disk write buffer area. When a write log IO request arrives, the IO scheduler puts it into the write log buffer. When the write log buffer is full, multiple write log IO requests are merged into a large write log IO request. When the operating status of the disk meets the redirection operation of the large write log IO request, the large write log IO request is placed in the disk request queue. Otherwise, the large write log IO request is placed in the solid-state drive request queue.When a disk write IO request arrives, the request directly enters the disk request queue. If the subsequent request is a request to write to the solid-state hard disk and the characteristics of the request do not meet the redirection conditions, the IO scheduler determines whether the length of the data to be written in the current disk write cache exceeds the threshold. If not, the request is placed in the disk request queue, otherwise it is placed in the solid-state hard disk request queue. Compared with the existing low-latency persistence strategy, the data carried by the write request is first written to the solid-state hard disk, and then the data is migrated from the solid-state hard disk to the disk, resulting in excessive wear of the solid-state hard disk and low disk utilization, which in turn leads to an imbalance in the data write between the solid-state hard disk and the disk in the storage node. The solution of the present application uses an IO scheduling strategy based on the operating status of the monitoring disk. When the operating status of the disk meets the redirection operation of the write request, the IO request originally sent to the solid-state hard disk whose request characteristics meet the redirection conditions is written to the disk. Under the premise of ensuring that the IO request meets the service quality requirements, it not only reduces the amount of write data of the solid-state hard disk, but also makes full use of the sequential write bandwidth of the disk, thereby effectively ensuring the balance of data write between the solid-state hard disk and the disk in the storage node. It can be understood that the above description is only exemplary, and the embodiments of the present application do not make any restrictions on this.

[0094] The storage node of this embodiment is used to implement the scheduling methods of the corresponding storage nodes in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here.

[0095] Reference Figure 5 , showing a schematic diagram of the structure of the scheduling device of the storage node in the fourth embodiment of the present application.

[0096] The scheduling device of the storage node of this embodiment includes: a first determination module 401, which is used to obtain characteristic data of the data write request and determine whether the characteristic data meets the configured write request redirection condition when the storage node receives a write request and determines that the write request is a data write request for the first storage device in the storage node; a second determination module 402, which is used to determine whether the operating state of the second storage device in the storage node is suitable for the redirection operation of the data write request if it is determined that the characteristic data meets the write request redirection condition, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; a first writing module 403, which is used to perform a redirection operation on the data write request if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, so as to write the data carried by the data write request into the second storage device.

[0097] The storage node scheduling device of this embodiment is used to implement the corresponding storage node scheduling methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0098] Reference Figure 6 , showing a schematic diagram of the structure of the scheduling device of the storage node in the fifth embodiment of the present application.

[0099] The scheduling device of the storage node of this embodiment includes: a first determination module 503, which is used to obtain characteristic data of the data write request and determine whether the characteristic data meets the configured write request redirection condition when the storage node receives a write request and determines that the write request is a data write request for the first storage device in the storage node; a second determination module 505, which is used to determine whether the operating state of the second storage device in the storage node is suitable for the redirection operation of the data write request if it is determined that the characteristic data meets the write request redirection condition, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; a first writing module 508, which is used to perform a redirection operation on the data write request if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, so as to write the data carried by the data write request into the second storage device.

[0100] Optionally, before the first determination module 503, the device also includes: a third determination module 501, which is used to mark the data write request as a redirection request if the length of the request queue of the first storage device corresponding to the acquired data write request exceeds a first preset length, and determine whether the operating state of the second storage device is suitable for the redirection operation of the data write request; a second writing module 502, which is used to perform a redirection operation on the data write request if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, so as to write the data carried by the data write request into the second storage device.

[0101] Optionally, the characteristic data includes at least one of the following: the size of the data carried by the data write request, the sender of the data write request, and the file type to which the data carried by the data write request belongs. Before the second determination module 505, the device also includes: a fourth determination module 504, which is used to determine that the characteristic data satisfies the write request redirection condition if it is determined that the size of the data carried by the data write request exceeds a configured threshold, and / or the sender of the data write request is a set sender, and / or the file type to which the data carried by the data write request belongs is a set file type.

[0102] Optionally, when the characteristic data includes the size of the data carried by the data write request, the device also includes: an analysis module 509, which is used to collect log data of the data write requests received by the storage node within a historical time period, and perform data analysis on the log data to obtain a static threshold for comparison with the size of the data carried by the data write request; an increase module 510, which is used to increase the static threshold according to a preset ratio if it is detected that the number of data write requests from the second storage device within the current time period exceeds a preset number, and the amount of data carried by the data write requests of the second storage device within the current time period exceeds a preset amount of data, so as to obtain a dynamic threshold for comparison with the size of the data carried by the data write request; a reduction module 511, which is used to reduce the static threshold according to a preset ratio if it is detected that the number of data write requests from the second storage device within the current time period does not exceed a preset number, and the amount of data carried by the data write requests of the second storage device within the current time period does not exceed a preset amount of data, so as to obtain a dynamic threshold for comparison with the size of the data carried by the data write request.

[0103] Optionally, the operating status of the second storage device includes at least one of the following: the length of the request queue of the second storage device, the size of the data to be written in the write cache area of ​​the second storage device, the utilization rate of the storage space of the second storage device, and the average time it takes for the second storage device to write data. Before the first writing module 508, the apparatus further includes: a fifth determination module 506, which is used to determine that the operating status of the second storage device is suitable for the redirection operation of the data write request if the result of the determination includes at least one of the following: the length of the request queue of the second storage device does not exceed the second preset length, the size of the data to be written in the write cache area of ​​the second storage device does not exceed the preset threshold, the utilization rate of the storage space of the second storage device does not exceed the preset utilization threshold, and the average time it takes for the second storage device to write data does not exceed the first preset time.

[0104] Optionally, when there are multiple second storage devices whose operating state is suitable for the redirection operation of the data write request, the device also includes: a first selection module 507, which is used to select the second storage device for writing the data carried by the data write request from multiple second storage devices by adopting a load balancing principle or a load concentration principle.

[0105] Optionally, the device also includes: a first cache module 512, which is used to cache the write log request in the log file of the write log cache area when the write request received by the storage node is a write log request of the first storage device; a merging module 513, which is used to perform a merging operation on the write log request in the log file of the write log cache area when the write log cache area is full, so as to obtain a large write log request; a third writing module 514, which is used to perform a redirection operation on the large write log request when it is determined that the operating state of the second storage device is suitable for the redirection operation of the large write log request, so as to write the log data carried by the large write log request to the second storage device.

[0106] Optionally, after the third writing module 514, the device further includes: a recording module 515, configured to record the file name of the log file to which the log writing request belongs and the offset of the log writing request in the log file to which it belongs.

[0107] The storage node scheduling device of this embodiment is used to implement the corresponding storage node scheduling methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0108] Reference Figure 7 , showing a schematic diagram of the structure of the scheduling device of the storage node in the sixth embodiment of the present application.

[0109] The scheduling device of the storage node of this embodiment includes: a first determination module 601, which is used to obtain characteristic data of the data write request and determine whether the characteristic data meets the configured write request redirection condition when the storage node receives a write request and determines that the write request is a data write request for the first storage device in the storage node; a second determination module 602, which is used to determine whether the operating state of the second storage device in the storage node is suitable for the redirection operation of the data write request if it is determined that the characteristic data meets the write request redirection condition, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; a first writing module 603, which is used to perform a redirection operation on the data write request if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, so as to write the data carried by the data write request into the second storage device.

[0110] Optionally, the device also includes: a second cache module 607, which is used to cache the write log request in the log file of the write log cache area when the write request received by the storage node is a write log request of the first storage device; a polling module 610, which is used to poll the status set by the assigned cache monitoring thread for the write log cache area; and a persistence module 612, which is used to perform persistent storage operations on the log data carried by the write log request in the log file of the write log cache area based on the status.

[0111] Optionally, before the polling module 610, the device also includes: a first update module 608, which is used to update the status of the write log cache area to a ready state using the cache monitoring thread assigned to the write log cache area if the write log cache area is full and the difference between the current time point and the time point when the write log cache area caches the first write log request is less than a second preset duration; and a second update module 609, which is used to update the status of the write log cache area to a dump state using the cache monitoring thread assigned to the write log cache area if the difference between the current time point and the time point when the write log cache area caches the first write log request is greater than or equal to the second preset duration.

[0112] Optionally, the persistence module 612 is specifically used to: when the state is the ready state, perform a merge operation on the write log requests in the log file of the write log cache area to obtain a large write log request; when it is determined that the operating state of the second storage device is suitable for the redirection operation of the large write log request, perform a redirection operation on the large write log request to write the log data carried by the large write log request to the second storage device.

[0113] Optionally, when there are multiple second storage devices whose operating state is suitable for the redirection operation of the large write log request, the device also includes: a second selection module 611, used to adopt a load balancing principle or a load concentration principle to select the second storage device for writing the log data carried by the large write log request from multiple second storage devices.

[0114] Optionally, the persistence module 612 is specifically used to: when the state is the dump state, perform a merge operation on the write log requests in the log file of the write log cache area to obtain a large write log request; adopt a load balancing principle or a load concentration principle to select the first storage device from multiple first storage devices for writing the log data carried by the large write log request.

[0115] Optionally, the device also includes: a sixth determination module 604, which is used to obtain characteristic data of the current write request and determine whether the characteristic data of the current write request meets the configured write request redirection condition when the current write request received by the storage node is a data write request for the first storage device and the previous write request received by the storage node is a data write request for the second storage device; a seventh determination module 605, which is used to determine whether the size of the to-be-written data in the write cache area of ​​the second storage device corresponding to the previous write request exceeds the configured threshold if it is determined that the characteristic data does not meet the write request redirection condition; and a fourth write module 606, which is used to perform a redirection operation on the current write request if it is determined that the size of the to-be-written data in the write cache area of ​​the second storage device corresponding to the previous write request does not exceed the configured threshold, so as to write the data carried by the current write request to the second storage device corresponding to the previous write request.

[0116] Optionally, the first writing module 603 includes: a storage submodule 6031, used to store the data write request in the request queue of the second storage device, and store the data carried by the data write request in the write cache area of ​​the second storage device; a writing submodule 6032, used to use a processing thread to poll the request queue of the second storage device to obtain the data write request in the request queue of the second storage device, and according to the size of the data carried by the data write request, obtain the data carried by the data write request from the write cache area of ​​the second storage device, and then write the data carried by the data write request into the file of the second storage device.

[0117] Optionally, after the writing submodule 6032, the first writing module 603 further includes: a recording submodule 6033, which is used to use the processing thread to record metadata of the data writing request.

[0118] Optionally, the metadata of the data write request includes at least one of the following: the size of the data carried by the data write request, the offset of the data carried by the data write request in the file to which it belongs, and the physical address of the second storage device. The recording submodule 6033 includes: a composition unit 6034, which is used to compose a new triplet of the size of the data carried by the data write request, the offset of the data carried by the data write request in the file to which it belongs, and the physical address of the second storage device; a determination unit 6035, which is used to query the hash table using the file name to which the data carried by the data write request belongs as a key to determine whether the key exists in the hash table; a first adding unit 6036, which is used to obtain the value corresponding to the key from the hash table if the key exists in the hash table, and obtain a linked list of the metadata of the file based on the value, and then add a new node with the value of the new triplet to the end of the linked list.

[0119] Optionally, the recording submodule 6033 also includes: a second adding unit 6037, used to create a new linked list if the key does not exist in the hash table, and add a new node with the value of the newly created triplet at the end of the new linked list; an inserting unit 6038, used to insert the head pointer of the new linked list as a value into the hash table, and insert the file name to which the data carried by the data write request belongs as a key into the hash table.

[0120] The storage node scheduling device of this embodiment is used to implement the corresponding storage node scheduling methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0121] Figure 8 This is a schematic diagram of the structure of an electronic device in Embodiment 7 of the present application; the electronic device may include:

[0122] One or more processors 701;

[0123] The computer readable medium 702 may be configured to store one or more programs,

[0124] When the one or more programs are executed by the one or more processors, the one or more processors implement the storage node scheduling method as described in the above-mentioned embodiment 1, embodiment 2, or embodiment 3.

[0125] Fig. 9 The hardware structure of the electronic device in the eighth embodiment of the present application; Figure 8 As shown, the hardware structure of the electronic device may include: a processor 801, a communication interface 802, a computer readable medium 803 and a communication bus 804;

[0126] The processor 801, the communication interface 802, and the computer readable medium 803 communicate with each other via a communication bus 804;

[0127] Optionally, the communication interface 802 may be an interface of a communication module, such as an interface of a GSM module;

[0128] Among them, the processor 801 can be specifically configured as follows: when a storage node receives a write request and determines that the write request is a data write request for a first storage device in the storage node, obtains characteristic data of the data write request and determines whether the characteristic data satisfies the configured write request redirection condition; if it is determined that the characteristic data satisfies the write request redirection condition, determines whether the operating status of the second storage device in the storage node is suitable for the redirection operation of the data write request, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; if it is determined that the operating status of the second storage device is suitable for the redirection operation of the data write request, performs a redirection operation on the data write request to write the data carried by the data write request to the second storage device.

[0129] The processor 801 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0130] The computer readable medium 803 can be, but is not limited to, a random access memory medium (RAM), a read-only memory medium (ROM), a programmable read-only memory medium (PROM), an erasable read-only memory medium (EPROM), an electrically erasable read-only memory medium (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0131] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program configured to be used by or in combination with an instruction execution system, an apparatus or a device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0132] Computer program code configured to perform the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0133] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. There are specific sequential relationships in the above-mentioned specific embodiments, but these sequential relationships are only exemplary. When the specific implementation is implemented, these steps may be less, more, or the execution order may be adjusted. That is, in some implementations as replacements, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0134] The modules involved in the embodiments of the present application may be implemented by software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor includes a first determination module, a second determination module, and a first writing module. The names of these modules do not constitute limitations on the modules themselves in certain circumstances. For example, the first determination module may also be described as "a module that obtains characteristic data of the data write request and determines whether the characteristic data satisfies the configured write request redirection conditions when the storage node receives a write request and determines that the write request is a data write request for the first storage device in the storage node."

[0135] As another aspect, the present application further provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the storage node scheduling method as described in the above-mentioned embodiment 1, embodiment 2, or embodiment 3.

[0136] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the device: when a storage node receives a write request and determines that the write request is a data write request of a first storage device in the storage node, obtains characteristic data of the data write request, and determines whether the characteristic data satisfies the configured write request redirection condition; if it is determined that the characteristic data satisfies the write request redirection condition, determines whether the operating state of the second storage device in the storage node is suitable for the redirection operation of the data write request, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, performs a redirection operation on the data write request to write the data carried by the data write request into the second storage device.

[0137] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0138] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0139] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A storage node scheduling method, the method comprising: When the storage node receives a write request and determines that the write request is a data write request of a first storage device in the storage node, obtains characteristic data of the data write request and determines whether the characteristic data satisfies a configured write request redirection condition; The characteristic data includes the size of the data carried by the data write request; If it is detected that the number of data write requests from the second storage device in the storage node in the current time period exceeds a preset number, and the amount of data carried by the data write requests from the second storage device in the current time period exceeds a preset amount of data, the static threshold is increased according to a preset ratio to obtain a dynamic threshold for comparison with the size of the data carried by the data write requests; The static threshold is obtained by analyzing log data of data write requests received by the storage node; If it is determined that the characteristic data satisfies the write request redirection condition, determining whether the operation state of the second storage device is suitable for the redirection operation of the data write request, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; If it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, a redirection operation is performed on the data write request to write the data carried by the data write request into the second storage device.

2. The method according to claim 1, wherein: Before obtaining the characteristic data of the data write request, the method further includes: If the length of the request queue of the first storage device obtained exceeds a first preset length, marking the data write request as a redirection request, and determining whether the operating state of the second storage device is suitable for the redirection operation of the data write request; If it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, a redirection operation is performed on the data write request to write the data carried by the data write request into the second storage device.

3. The method according to claim 1, wherein: The characteristic data also includes at least one of the following: the sender of the data write request, the file type to which the data carried by the data write request belongs, Before determining that the characteristic data satisfies the write request redirection condition, the method further includes: If it is determined that the size of the data carried by the data write request exceeds the configured threshold, and / or the sender of the data write request is the set sender, and / or the file type to which the data carried by the data write request belongs is the set file type, then it is determined that the characteristic data meets the write request redirection condition.

4. The method according to claim 3, wherein: When the characteristic data includes the size of the data carried by the data write request, the method further includes: Collecting log data of data write requests received by the storage node within a historical time period, and performing data analysis on the log data to obtain a static threshold value for comparison with the size of data carried by the data write request; If it is detected that the number of data write requests from the second storage device in the current time period does not exceed the preset number, and the amount of data carried by the data write requests from the second storage device in the current time period does not exceed the preset amount of data, the static threshold is reduced by a preset ratio to obtain a dynamic threshold for comparison with the size of the data carried by the data write requests.

5. The method according to claim 1, wherein: The operation status of the second storage device includes at least one of the following: the length of the request queue of the second storage device, the size of the to-be-written data in the write buffer area of ​​the second storage device, the utilization rate of the storage space of the second storage device, and the average time for the second storage device to write data. Before determining that the operating state of the second storage device is suitable for the redirection operation of the data write request, the method further includes: If the determined result includes at least one of the following: the length of the request queue of the second storage device does not exceed the second preset length, the size of the data to be written in the write cache area of ​​the second storage device does not exceed the preset threshold, the utilization rate of the storage space of the second storage device does not exceed the preset utilization threshold, and the average time for the second storage device to write data does not exceed the first preset time, then it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request.

6. The method according to claim 1, wherein: When the number of the second storage devices whose operating status is suitable for the redirection operation of the data write request is multiple, the method further includes: The load balancing principle or the load concentration principle is adopted to select the second storage device for writing the data carried by the data write request from a plurality of the second storage devices.

7. The method according to claim 1, wherein: The method further comprises: When the write request received by the storage node is a write log request of the first storage device, cache the write log request in a log file in the write log cache area; When the write log buffer is full, performing a merge operation on the write log requests in the log files of the write log buffer to obtain a large write log request; When it is determined that the operating state of the second storage device is suitable for the redirection operation of the large write log request, a redirection operation is performed on the large write log request to write the log data carried by the large write log request into the second storage device.

8. The method according to claim 7, wherein: After storing the write log request in the log file of the write log buffer area, the method further includes: The file name of the log file to which the log write request belongs and the offset of the log write request in the log file to which it belongs are recorded.

9. The method according to claim 1, wherein: The method further comprises: When the write request received by the storage node is a write log request of the first storage device, cache the write log request in a log file in the write log cache area; The cache monitoring thread assigned by polling is set for the state of the write log cache area; Based on the state, a persistent storage operation is performed on the log data carried by the write log request in the log file of the write log cache area.

10. The method according to claim 9, wherein: Before the cache monitoring thread assigned by the polling is set for the state of the write log cache area, the method further includes: If the write log cache is full, and the difference between the current time point and the time point when the write log cache cached the first write log request is less than a second preset duration, the cache monitoring thread allocated to the write log cache is used to update the state of the write log cache to a ready state; If the difference between the current time point and the time point when the write log cache cached the first write log request is greater than or equal to a second preset duration, the cache monitoring thread allocated to the write log cache is used to update the state of the write log cache to a dump state.

11. The method according to claim 10, wherein: The performing a persistent storage operation on the log data carried by the write log request in the log file of the write log buffer area based on the state includes: When the state is the ready state, performing a merge operation on the write log requests in the log file of the write log buffer area to obtain a large write log request; When it is determined that the operating state of the second storage device is suitable for the redirection operation of the large write log request, a redirection operation is performed on the large write log request to write the log data carried by the large write log request into the second storage device.

12. The method according to claim 11, wherein: When the number of the second storage devices whose operating status is suitable for the redirection operation of the large write log request is multiple, the method further includes: The load balancing principle or the load concentration principle is adopted to select the second storage device for writing the log data carried by the large log write request from the plurality of second storage devices.

13. The method according to claim 10, wherein: The performing a persistent storage operation on the log data carried by the write log request in the log file of the write log buffer area based on the state includes: When the state is the dump state, performing a merge operation on the write log request in the log file of the write log buffer area to obtain a large write log request; The first storage device for writing the log data carried by the large log write request is selected from a plurality of the first storage devices by adopting a load balancing principle or a load concentration principle.

14. The method according to claim 1, wherein: The method further comprises: When the current write request received by the storage node is a data write request for the first storage device, and the previous write request received by the storage node is a data write request for the second storage device, acquiring characteristic data of the current write request, and determining whether the characteristic data of the current write request meets the configured write request redirection condition; If it is determined that the characteristic data does not satisfy the write request redirection condition, determining whether the size of the to-be-written data in the write cache area of ​​the second storage device corresponding to the previous write request exceeds a configured threshold; If it is determined that the size of the data to be written in the write cache area of ​​the second storage device corresponding to the previous write request does not exceed the configured threshold, a redirection operation is performed on the current write request to write the data carried by the current write request to the second storage device corresponding to the previous write request.

15. The method according to claim 1, wherein: The performing a redirection operation on the data write request to write the data carried by the data write request into the second storage device includes: storing the data write request in a request queue of the second storage device, and storing the data carried by the data write request in a write buffer area of ​​the second storage device; Use a processing thread to poll the request queue of the second storage device to obtain the data write request in the request queue of the second storage device, and obtain the data carried by the data write request from the write cache area of ​​the second storage device according to the size of the data carried by the data write request, and then write the data carried by the data write request into the file of the second storage device.

16. The method according to claim 15, wherein: After writing the data carried by the data write request into the second storage device, the method further includes: Using the processing thread, metadata of the data write request is recorded.

17. The method according to claim 16, wherein: The metadata of the data write request includes at least one of the following: The size of the data carried by the data write request, the offset of the data carried by the data write request in the corresponding file, and the physical address of the second storage device. The using the processing thread to record metadata of the data write request includes: Create a new triplet by combining the size of the data carried by the data write request, the offset of the data carried by the data write request in the corresponding file, and the physical address of the second storage device; Using the file name of the data carried in the data write request as a key to query the hash table to determine whether the key exists in the hash table; If the key exists in the hash table, the value corresponding to the key is obtained from the hash table, and based on the value, a linked list of metadata of the file is obtained, and then a new node with the value of the newly created triple is added to the end of the linked list.

18. The method according to claim 17, wherein: The method further comprises: If the key does not exist in the hash table, a new linked list is created, and a new node whose value is the newly created triple is added to the tail of the new linked list; The head pointer of the new linked list is inserted into the hash table as a value, and the file name to which the data carried by the data write request belongs is inserted into the hash table as a key.

19. A storage node scheduling device, the device comprising: A first determination module is used to, when a storage node receives a write request and determines that the write request is a data write request for a first storage device in the storage node, obtain characteristic data of the data write request and determine whether the characteristic data satisfies a configured write request redirection condition; The characteristic data includes the size of the data carried by the data write request; an increasing module, configured to increase the static threshold value according to a preset ratio to obtain a dynamic threshold value for comparison with the size of the data carried by the data write request, if it is detected that the number of data write requests from the second storage device in the storage node in the current time period exceeds a preset number, and the amount of data carried by the data write request from the second storage device in the current time period exceeds a preset amount of data; The static threshold is obtained by analyzing log data of data write requests received by the storage node; a second determination module, configured to determine whether the operating state of the second storage device is suitable for the redirection operation of the data write request if it is determined that the characteristic data satisfies the write request redirection condition, wherein the read and write performance of the first storage device is better than the read and write performance of the second storage device; The first writing module is used to perform a redirection operation on the data writing request if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data writing request, so as to write the data carried by the data writing request into the second storage device.

20. The device according to claim 19, wherein Before the first determining module, the device further includes: a third determining module, configured to mark the data write request as a redirection request if the length of the request queue of the first storage device obtained exceeds a first preset length, and determine whether the operation state of the second storage device is suitable for the redirection operation of the data write request; The second writing module is used to perform a redirection operation on the data write request if it is determined that the operating state of the second storage device is suitable for the redirection operation of the data write request, so as to write the data carried by the data write request into the second storage device.

21. The device according to claim 19, wherein The characteristic data also includes at least one of the following: the sender of the data write request, the file type to which the data carried by the data write request belongs, Before the second determining module, the device further includes: The fourth determination module is used to determine that the characteristic data meets the write request redirection condition if it is determined that the size of the data carried by the data write request exceeds a configured threshold, and / or the sender of the data write request is a set sender, and / or the file type to which the data carried by the data write request belongs is a set file type.

22. The device according to claim 19, wherein The operation status of the second storage device includes at least one of the following: the length of the request queue of the second storage device, the size of the to-be-written data in the write buffer area of ​​the second storage device, the utilization rate of the storage space of the second storage device, and the average time for the second storage device to write data. Before the first writing module, the device further includes: A fifth determination module is used to determine that the operating state of the second storage device is suitable for the redirection operation of the data write request if the determination result includes at least one of the following: the length of the request queue of the second storage device does not exceed the second preset length, the size of the data to be written in the write cache area of ​​the second storage device does not exceed the preset threshold, the utilization rate of the storage space of the second storage device does not exceed the preset utilization rate threshold, and the average time for the second storage device to write data does not exceed the first preset time.

23. The device according to claim 19, wherein: When the number of the second storage devices whose operating status is suitable for the redirection operation of the data write request is multiple, the apparatus further includes: The first selection module is used to select the second storage device for writing the data carried by the data write request from a plurality of the second storage devices by adopting a load balancing principle or a load concentration principle.

24. The device according to claim 19, wherein The device also includes: A first cache module, configured to cache the write log request in a log file in a write log cache area when the write request received by the storage node is a write log request of the first storage device; A merging module, configured to, when the write log buffer is full, perform a merging operation on the write log requests in the log files of the write log buffer to obtain a large write log request; The third writing module is used to perform a redirection operation on the large write log request when it is determined that the operating state of the second storage device is suitable for the redirection operation of the large write log request, so as to write the log data carried by the large write log request into the second storage device.

25. The device according to claim 19, wherein The device also includes: A second cache module, configured to cache the write log request in a log file in the write log cache area when the write request received by the storage node is a write log request of the first storage device; A polling module, used for polling the status set by the assigned cache monitoring thread for the write log cache area; A persistence module is used to perform a persistent storage operation on the log data carried by the write log request in the log file of the write log buffer area based on the state.

26. The device according to claim 19, wherein The device also includes: a sixth determination module, configured to obtain characteristic data of the current write request and determine whether the characteristic data of the current write request satisfies a configured write request redirection condition when the current write request received by the storage node is a data write request for the first storage device and the previous write request received by the storage node is a data write request for the second storage device; a seventh determination module, configured to determine whether the size of the to-be-written data in the write cache area of ​​the second storage device corresponding to the previous write request exceeds a configured threshold if it is determined that the characteristic data does not satisfy the write request redirection condition; The fourth writing module is used to perform a redirection operation on the current write request if it is determined that the size of the to-be-written data in the write cache area of ​​the second storage device corresponding to the previous write request does not exceed the configured threshold, so as to write the data carried by the current write request to the second storage device corresponding to the previous write request.

27. The device according to claim 19, wherein The first writing module includes: a storage submodule, used to store the data write request in the request queue of the second storage device, and store the data carried by the data write request in the write buffer area of ​​the second storage device; A write submodule is used to use a processing thread to poll the request queue of the second storage device to obtain the data write request in the request queue of the second storage device, and obtain the data carried by the data write request from the write cache area of ​​the second storage device according to the size of the data carried by the data write request, and then write the data carried by the data write request into the file of the second storage device.

28. A storage node, comprising: A scheduler provided in the storage node, The scheduler is specifically used for: When the storage node receives a write request and determines that the write request is a data write request for a solid state drive in the storage node, obtains characteristic data of the data write request and determines whether the characteristic data satisfies a configured write request redirection condition; The characteristic data includes the size of the data carried by the data write request; If it is detected that the number of data write requests from the second storage device in the storage node in the current time period exceeds a preset number, and the amount of data carried by the data write requests from the second storage device in the current time period exceeds a preset amount of data, the static threshold is increased according to a preset ratio to obtain a dynamic threshold for comparison with the size of the data carried by the data write requests; The static threshold is obtained by analyzing log data of data write requests received by the storage node; If it is determined that the characteristic data meets the write request redirection condition, determining whether the operation state of the disk in the storage node is suitable for the redirection operation of the data write request; If it is determined that the operating state of the disk is suitable for the redirection operation of the data write request, a redirection operation is performed on the data write request to write the data carried by the data write request into the disk.

29. An electronic device, comprising: one or more processors; A computer readable medium configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the storage node scheduling method as described in any one of claims 1 to 18.

30. A computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for scheduling a storage node according to any one of claims 1 to 18 is implemented.

Images