Data processing method and apparatus, and electronic device

Abstract

Embodiments of the present application provide a data processing method and device and electronic equipment. In the embodiments of the present application, when node expansion occurs in a distributed storage system, a virtual node migration subset corresponding to the node is first determined, and then for a received write data request in which the corresponding virtual node is not in the virtual node migration subset, the write data request is added to a second processing linked list for normal processing without waiting for the virtual node to complete disk flushing before processing, which can effectively avoid the performance decline problem of the distributed storage system when node expansion occurs, and improve the data processing efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a data processing method and device and electronic equipment. BACKGROUND

[0002] When the use capacity of a distributed storage system reaches a certain threshold, data writing cannot be performed. Therefore, in order to maintain the availability of the distributed storage system, node expansion needs to be performed. Node expansion causes the mapping relationship between virtual nodes and physical nodes to change, that is, some virtual nodes are mapped to new physical nodes. In order to ensure that the data of the virtual nodes mapped to the new physical nodes does not appear in the original corresponding physical nodes after the node expansion is completed, and to ensure data consistency, the data of the virtual nodes in the cache needs to be flushed to the disk, and the corresponding log records need to be deleted.

[0003] At present, in actual applications, when node expansion occurs, the received write data request is inserted into a cache queue, and after the data of the virtual nodes in the cache is flushed to the disk, the write data request in the cache queue is directly transmitted to a write-time redirection module for write-time redirection processing. However, in the above process, if the flushing time is long, the received write data request will not be processed for a long time, which will cause the performance of the distributed storage system to decrease. SUMMARY

[0004] Therefore, the present application provides a data processing method and device and electronic equipment to avoid the performance decrease of the distributed storage system when node expansion occurs.

[0005] The present application provides a data processing method, which is applied to any specified storage node in a distributed storage system; the specified storage node refers to a storage node included in the distributed storage system before this node expansion; the method comprises the following steps:

[0006] A virtual node migration set is obtained, a virtual node migration subset corresponding to the node is determined according to the virtual node migration set, and information of at least one data processing request received before a current time point and not processed is recorded in a first processing chain table; the virtual node migration set is composed of at least one virtual node mapped to a new storage node during this node expansion;

[0007] The receiving module is configured to receive a data processing request, and when the data processing request is a write data request, the write data request indicates a node identifier corresponding to a virtual node matching a storage location to which data is to be written; if it is determined that the virtual node corresponding to the node identifier is in the virtual node migration subset, the write data request is added to a cache queue for caching write data requests requiring write-time redirection processing; otherwise, information of the write data request is added to the second processing chain table until it is found that all data processing requests in the first processing chain table are processed.

[0008] When it is found that all write data requests in the second processing chain table are processed, the cache data corresponding to the write data requests in the current flush chain table is flushed until a set flush stop condition is reached; the set flush stop condition refers to that a target log object is deleted, and the target log object records an operation log corresponding to a last processed write data request in the first processing chain table.

[0009] Embodiments of the present application also provide a data processing device, which is configured in any specified storage node in a distributed storage system; the specified storage node refers to a storage node included in the distributed storage system before the current node expansion; the device comprises:

[0010] The obtaining module is configured to obtain a virtual node migration set, determine a virtual node migration subset corresponding to the current node according to the virtual node migration set, and record information of at least one data processing request received before a current time point and not processed to a first processing chain table; the virtual node migration set is composed of at least one virtual node mapped to a newly added storage node during the current node expansion;

[0011] The receiving module is configured to receive a data processing request, and when the data processing request is a write data request, the write data request indicates a node identifier corresponding to a virtual node matching a storage location to which data is to be written; if it is determined that the virtual node corresponding to the node identifier is in the virtual node migration subset, the write data request is added to a cache queue for caching write data requests requiring write-time redirection processing; otherwise, information of the write data request is added to the second processing chain table until it is found that all data processing requests in the first processing chain table are processed;

[0012] The flush module is configured to, when it is found that all write data requests in the second processing chain table are processed, flush cache data corresponding to the write data requests in the current flush chain table until a set flush stop condition is reached; the set flush stop condition refers to that a target log object is deleted, and the target log object records an operation log corresponding to a last processed write data request in the first processing chain table.

[0013] Embodiments of the present application also provide an electronic device, which comprises:

[0014] a processor; and

[0015] a memory having stored therein computer program instructions, which, when executed by the processor, cause the processor to perform the steps of the method as above.

[0016] The application also provides a computer readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the method as above.

[0017] From the above technical solutions, it can be seen that in the embodiments of the application, when node expansion occurs in the distributed storage system, the virtual node migration subset corresponding to the node is first determined, and then for the received write data request of the corresponding virtual node not in the virtual node migration subset, the write data request is added to the second processing linked list for normal processing without waiting for the virtual node to complete the disk flushing before processing, which can effectively avoid the performance decline problem of the distributed storage system when node expansion occurs, and improve the data processing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0019] Figure 1 Method flowchart provided for the embodiments of the application.

[0020] Figure 2 Interactive flowchart of the data processing method provided for the embodiments of the application.

[0021] Figure 3 Cache processing schematic diagram provided for the embodiments of the application.

[0022] Figure 4 Another cache processing schematic diagram provided for the embodiments of the application.

[0023] Figure 5 Another cache processing schematic diagram provided for the embodiments of the application.

[0024] Figure 6 Another cache processing schematic diagram provided for the embodiments of the application.

[0025] Figure 7 Another cache processing schematic diagram provided for the embodiments of the application.

[0026] Figure 8 Another cache processing schematic diagram provided for the embodiments of the application.

[0027] Figure 9An apparatus structure schematic diagram provided by the embodiment of the present application.

[0028] Figure 10 An electronic device structure schematic diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0029] For the convenience of understanding the present scheme, before describing the present scheme, the technical problems existing in the prior art distributed storage system related by the embodiment of the present application are described:

[0030] With the passage of time, the available capacity of the distributed storage system gradually decreases, and when the used capacity of the distributed storage system reaches a certain threshold, the data writing cannot be performed. Therefore, in order to maintain the availability of the distributed storage system, the node expansion (i.e. adding new physical nodes, which can be considered as storage nodes) needs to be performed. The distributed storage system usually uses cache technology to improve performance. In the specific implementation, for example, for the data processing request from the client, an operation log corresponding to the data processing request is first recorded (the operation log may include the operation type, data content, and storage location to be operated of the data processing request, etc.). Then, if the data processing request is a write data request, the data to be written corresponding to the write data request is written into the cache (here, the data to be written into the cache corresponding to the write data request can be recorded as the cache data corresponding to the write data request), the write data request is inserted into the disk flushing link table, and the response is returned to the client to indicate that the write data request processing is completed. If the data processing request is a read data request, the data to be read corresponding to the read data request is directly returned to the client. The above data cache mode can also be considered as a normal mode.

[0031] The node expansion causes the mapping relationship between the virtual node and the physical node to change, that is, part of the virtual nodes are mapped to the new physical node, and these virtual nodes mapped to the new physical node can be recorded as migrated virtual nodes. To ensure that the data of the migrated virtual nodes does not appear in the original corresponding physical node after the node expansion is completed, to ensure the consistency of the data, the data of the migrated virtual nodes in the cache needs to be flushed, and the corresponding log record is deleted. At present, when the node expansion occurs, the write data request received is usually processed in a full transparent mode, for example, the write data request is inserted into a cache queue, and after the data of the migrated virtual node in the cache is flushed, the write data request (which can be recorded as a write IO request) in the cache queue is directly transmitted to the write-time redirection module for write-time redirection processing, and all subsequent new write IO requests are also directly transmitted to the write-time redirection module for processing. However, in the above process, if the flushing time is long, the received write data request will not be processed for a long time, that is, there will be a long IO drop zero time; and after the flushing is completed, the write IO request is directly transmitted to the write-time redirection module for processing, which also causes the performance of the distributed storage system to decrease.

[0032] Among them, the above-mentioned Redirect On Write (ROW) refers to a technology of converting random write into sequential write and aggregating small IO into large IO, thereby improving the system write performance. The basic implementation principle of the write-time redirection is as follows: a plurality of small IOs are first stored in a buffer, and when the buffer reaches a certain capacity, they are written to a persistent storage medium (such as a disk) at one time. The above-mentioned virtual node refers to that in a distributed storage system, when data is distributed to different physical nodes, a hash algorithm is usually used to map the key of the data to a virtual node, and then the virtual node is mapped to a specific physical node; in this way, the uniform distribution of data between physical nodes can be ensured, thereby improving the performance and scalability of the system. Here, the key of the data can be the storage location of the data operation (such as a logical address) and the like.

[0033] To solve the above technical problems, the embodiment of the present application provides a data processing method. In order to make the technical personnel in the art better understand the technical solutions provided by the embodiments of the present application, and make the above-mentioned purposes, characteristics and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the drawings.

[0034] Referring to Figure 1 , Figure 1A method flowchart is provided for an embodiment of the present application. The method is applied to any specified storage node in a distributed storage system; here, the specified storage node refers to a storage node included in the distributed storage system before the current node expansion. As an embodiment, the storage node may refer to an electronic device such as a server.

[0035] As shown in Figure 1 the flowchart may include the following steps:

[0036] In step 101, a virtual node migration set is obtained, a virtual node migration subset corresponding to the current node is determined according to the virtual node migration set, and information of at least one data processing request received before the current time point and not processed is recorded in a first processing chain table.

[0037] In the embodiment, the virtual node migration set is composed of at least one virtual node (denoted as a migration virtual node) mapped to the newly added storage node during the current node expansion.

[0038] As an embodiment, the virtual node migration set in this step is determined by the target storage node based on all pre-configured virtual nodes and all storage nodes included in the distributed storage system after the current node expansion; here, the target storage node refers to the specified storage node that detects the node expansion event. After determining the virtual node migration set, the target storage node sends the virtual node migration set to each specified storage node, and then the current node can obtain the virtual node migration set and determine the virtual node migration subset corresponding to the current node according to the virtual node migration set. As for how to determine the virtual node migration set based on all pre-configured virtual nodes and all storage nodes included in the distributed storage system after the current node expansion, examples will be described below, which will not be described here.

[0039] In the embodiment, the detection of the node expansion event in the specific implementation includes: when receiving the node expansion indication sent by the client, it is determined that the node expansion event is detected. The node expansion indication here is initiated by the client when receiving the node expansion instruction triggered by the user on the client interface.

[0040] As an embodiment, the determination of the virtual node migration subset corresponding to the current node according to the virtual node migration set in the specific implementation may be, for example: first, obtain the virtual node mapping table in the local storage, which indicates at least one virtual node mapped by the current node before the current node expansion; then, for each virtual node in the virtual node mapping table, check whether the virtual node is in the virtual node migration set according to the node identifier (such as node number, etc.) corresponding to the virtual node, if yes, determine that the virtual node is a migration virtual node corresponding to the current node; finally, based on all the determined migration virtual nodes, the virtual node migration subset corresponding to the current node is constructed.

[0041] In the embodiment, the current time point can be considered as a time point when the virtual node migration subset corresponding to the node is determined. Based on this, the data processing request received before the current time point and not processed completely can be considered as a data processing request received before the current time point and not returned to the client for response. Alternatively, the data processing request can include a read data request and a write data request.

[0042] In step 102, a data processing request is received. When the data processing request is a write data request, the write data request indicates a node identifier corresponding to a virtual node matching a storage location to which data is to be written. If it is determined that the virtual node corresponding to the node identifier is in the virtual node migration subset, the write data request is added to a cache queue for caching write data requests requiring write-time redirection processing. Otherwise, information of the write data request is added to a second processing chain table until it is found that all data processing requests in the first processing chain table are processed completely.

[0043] In the embodiment, for a data processing request received after the virtual node migration subset corresponding to the node is determined, when the data processing request is a write data request, if it is determined that the virtual node corresponding to the node identifier indicated by the data request is in the virtual node migration subset, the write data request is added to the cache queue for caching write data requests requiring write-time redirection processing. Otherwise, information of the write data request is added to the second processing chain table until it is found that all data processing requests in the first processing chain table are processed completely.

[0044] It should be noted that the data processing requests in the first processing chain table and the second processing chain table are processed normally in the order of writing to the chain table. Here, normal processing refers to cache processing according to the cache technique method described above.

[0045] It should be further noted that when it is found that all data processing requests in the first processing chain table are processed completely, the addition of information of a received write data request to the second processing chain table is stopped to avoid the second processing chain table from being cleared forever. Then, in the embodiment, a read data request received after it is found that all data processing requests in the first processing chain table are processed completely is processed normally. For a received write data request, it is determined whether the virtual node corresponding to the write data request is in the virtual node migration set. If yes, the write data request is added to the cache queue for caching write data requests requiring write-time redirection processing. Otherwise, information of the write data request is added to a preconfigured cache processing chain table for cache processing.

[0046] Step 103, when it is found that all the write data requests in the second processing chain table are processed, the cache data corresponding to the write data request in the current disk flushing chain table is flushed until the set disk flushing stop condition is reached.

[0047] In the embodiment, when it is found that all the write data requests in the second processing chain table are processed, it can be indicated that all the write data requests in the current first processing chain table have been inserted into the disk flushing chain table to wait for disk flushing processing. At this time, the cache data corresponding to the write data request in the current disk flushing chain table can be flushed until the set disk flushing stop condition is reached.

[0048] In the embodiment, the disk flushing mentioned above refers to writing the cache data into a persistent storage medium (such as a disk, etc.). The set disk flushing stop condition refers to that the target log object is deleted, and the target log object records the operation log corresponding to the last processed write data request in the first processing chain table. Here, the target log object is deleted (i.e., the disk flushing stop condition is reached), which can indicate that the cache data corresponding to all the write data requests in the first processing chain table has been flushed. As for how to determine the target log object, examples will be described below, which will not be described here.

[0049] At this point, the process shown in Figure 1 is completed.

[0050] As can be seen from the process shown in Figure 1 , in the embodiment of the present application, when the node expansion occurs in the distributed storage system, the virtual node migration subset corresponding to the node is first determined, and then for the received write data request whose corresponding virtual node is not in the virtual node migration subset, the write data request is added to the second processing chain table to wait for normal processing without waiting for the virtual node to be flushed before processing. This can effectively avoid the performance decline problem of the distributed storage system when the node expansion occurs, and improve the data processing efficiency.

[0051] The following describes how to determine the virtual node migration set based on all the preconfigured virtual nodes and all the storage nodes included in the distributed storage system after the current node expansion in step 101:

[0052] As an embodiment, there are many ways to determine the virtual node migration set based on all the pre-configured virtual nodes and all the storage nodes included in the distributed storage system after this time of node expansion, such as, first, determining at least one virtual node corresponding to each storage node based on all the pre-configured virtual nodes and all the storage nodes included in the distributed storage system after this time of node expansion, and recording a virtual node mapping table composed of at least one virtual node corresponding to each storage node as a new virtual node mapping table corresponding to the storage node; then, determining the virtual nodes corresponding to each newly added storage node after this time of node expansion as the migration virtual nodes; finally, constructing the virtual node migration set based on the determined migration virtual nodes.

[0053] In the embodiment, the specific implementation of determining at least one virtual node corresponding to each storage node based on all the pre-configured virtual nodes and all the storage nodes included in the distributed storage system after this time of node expansion may be, for each virtual node in all the virtual nodes, first, performing a remainder operation on the node identifier (such as node number, etc.) of the virtual node and the total number of storage nodes to obtain a remainder result, such as the remainder result of the node identifier 1 and the total number of storage nodes 4 is 1; then, determining the storage node corresponding to the storage node identifier matching the remainder result as the storage node to which the virtual node is mapped, such as the storage node 1 matching the remainder result 1 is the storage node to which the virtual node 1 is mapped.

[0054] The virtual node migration subset of the node in the step 101 is described as follows:

[0055] As an embodiment, the node is pre-configured with N threads for processing data processing requests, N is greater than 1, for concurrent processing of data processing requests and improving the efficiency of data processing. Each thread corresponds to a partial virtual node migration subset; the partial virtual node migration subsets corresponding to the threads are different; and the partial virtual node migration subsets corresponding to the threads constitute the virtual node migration subset.

[0056] In the embodiment, there are many specific implementation ways to determine the partial virtual node migration subset corresponding to each thread, such as, for each migration virtual node in the virtual node migration subset corresponding to the node, first, calculating the remainder value between the node identifier corresponding to the migration virtual node and the total number of threads, then, determining the thread corresponding to the thread identifier (such as thread number, etc.) matching the remainder value as the thread to which the migration virtual node is allocated; finally, constructing the partial virtual node migration subset corresponding to each thread based on at least one migration virtual node allocated to each thread.

[0057] Based on the above description, as an embodiment, after receiving a data processing request, the node assigns the data processing request to a thread matched with the virtual node identifier indicated by the data processing request for processing based on the virtual node identifier.

[0058] The following describes how to determine the target log object in step 103:

[0059] As an embodiment, the determination of the target log object can be implemented as follows: when it is found that all data processing requests in the first processing chain table have been processed, it indicates that the operation logs corresponding to the write data requests in the first processing chain table have been recorded in the corresponding log objects, and therefore, the currently used log object can be obtained from the configured log objects for recording the operation logs corresponding to the write data requests as the target log object, which definitely records the operation log corresponding to the last processed write data request in the first processing chain table.

[0060] As an embodiment, for any used log object, the used log object can be deleted only after all cache data corresponding to the write data requests corresponding to the operation logs in the used log object have been flushed. Therefore, if the used log object records less operation logs, the used log object is deleted earlier. Based on this, in order to ensure that the target log object can be deleted as soon as possible to achieve the flushing stop condition as soon as possible, the target log object can be stopped from being used after the target log object is determined, so as to avoid that new operation logs are recorded in the target log object, and the next unused log object in the configured log objects is used as the currently used log object.

[0061] The following describes the log object processing operation after the flushing of the cache data corresponding to each write data request in the current flushing chain table is completed in step 103:

[0062] As an embodiment, for each write data request in the current flushing chain table, after the flushing of the cache data corresponding to the write data request is completed, it can be checked whether the log object recording the operation log corresponding to the write data request still contains the operation log corresponding to the write data request that has not been flushed; if not, the log object recording the operation log corresponding to the write data request can be deleted.

[0063] The following further describes the processing process after the set flushing stop condition is reached in step 103:

[0064] As an example, when the set disk flushing stop condition is found to be met, it can be determined that the cache data corresponding to each virtual node in the virtual node migration subset has been flushed to disk, and the disk flushing completion event indicating that the cache data has been flushed to disk is reported to the target storage node.

[0065] Upon receiving disk flush completion events reported by all designated storage nodes, the target storage node sends a virtual node mapping table update event to each storage node included in the distributed storage system after this node expansion. This instructs the storage node to update its local historical virtual node mapping table to the new virtual node mapping table indicated by the update event. The new virtual node mapping table is determined based on all pre-configured virtual nodes and all storage nodes included in the distributed storage system after this node expansion.

[0066] As for how to determine the new virtual node mapping table based on all pre-configured virtual nodes and all storage nodes included in the distributed storage system after this node expansion, please refer to the relevant description of determining the virtual node migration set above, which will not be repeated here.

[0067] To facilitate understanding of the specific implementation process of the above data processing method, specific examples are provided below.

[0068] See Figure 2 As shown, the data processing method includes the following steps:

[0069] S201, the main monitoring service module sends a virtual node migration set to the data processing service module of each specified storage node.

[0070] In this embodiment, the node with the main monitoring service module is the aforementioned target storage node. Here, when the node with the main monitoring service module detects a node expansion event, it generates a corresponding virtual node migration set and sends the virtual node migration set to the data processing service instance of each specified storage node.

[0071] The node with the main monitoring service module deployed here refers to the target storage node mentioned above. This embodiment does not limit the specific functions of the main monitoring service module; for example, it can be used to detect whether there are expansion events, generate virtual node migration sets, collect disk flushing completion events, etc.

[0072] S202, the data processing service module processes the received virtual node migration set and sends the virtual node migration set to the cache control module.

[0073] Here, there are no specific restrictions on how the data processing service instance handles the received virtual node migration set. For example, the virtual node migration set can be recorded in a specified data structure.

[0074] S203, the cache control module calculates a virtual node migration subset based on the virtual node migration set, and uses the first IO count to count the in-flight IO request.

[0075] Here, the in-flight IO request refers to an IO request received before the current time point but not processed completely. The IO request here can be considered as the data processing request described above.

[0076] S204, the cache control module switches the cache state to a waiting first IO count zero state.

[0077] The cache control module switches the cache state from the default state to the waiting first IO count zero state to wait for the first IO count to be zero. The default state here can be considered as the cache state corresponding to the normal mode described above. For the specific implementation principle of the normal mode, please refer to the description above, which will not be described here.

[0078] S205, the cache control module processes the write IO request based on the judgment result of whether the virtual node corresponding to the received write IO request is in the virtual node migration subset.

[0079] In specific implementation, for example, the cache control module receives an IO request. If the IO request is a write IO request, it is judged whether the virtual node corresponding to the virtual node identifier indicated by the write IO request is in the virtual node migration subset. If yes, the write IO request is added to a specified cache queue to be transmitted to the write-time redirection module for processing after the subsequent disk flushing is completed. Otherwise, the write IO request is counted on the second IO count.

[0080] Here, the specified cache queue refers to the cache queue for caching the write data request requiring write-time redirection processing. The write IO request received after the state switching using the second IO count is to avoid the first IO count from being unable to be zeroed all the time.

[0081] S206, the cache control module obtains the currently used log object from the cache log module as a target log object after finding that the first IO count is zero.

[0082] S207, the cache control module notifies the cache log module to replace the currently used log object with the next unused log object.

[0083] S208, the cache control module switches the cache state to a waiting second IO count zero state.

[0084] The cache control module switches the cache state from the waiting first IO count zero state to the waiting second IO count zero state to wait for the second IO count to be zero.

[0085] S209, the cache control module processes the write IO request based on the result of determining whether the virtual node corresponding to the virtual node identifier indicated by the write IO request is in the virtual node migration subset.

[0086] In a specific implementation, for example, the cache control module receives an IO request. If the IO request is a write IO request, the cache control module determines whether the virtual node corresponding to the virtual node identifier indicated by the write IO request is in the virtual node migration subset. If yes, the cache control module adds the write IO request to the specified cache queue, so as to be transmitted to the write-time redirection module for processing after the subsequent disk flushing is completed. Otherwise, the cache control module counts the write IO request on the first IO counter.

[0087] Here, the write IO request received after the first IO counter state is switched is continuously counted, so as to avoid that the second IO counter cannot be cleared all the time.

[0088] S210, after the cache control module finds that the second IO counter is cleared, the cache control module notifies the cache flushing module to switch the disk flushing mode to the forced disk flushing mode.

[0089] After the cache control module finds that the second IO counter is cleared, the cache control module notifies the cache flushing module to switch the disk flushing mode to the forced disk flushing mode, so as to perform forced disk flushing on the disk flushing units in the current disk flushing linked list. The disk flushing unit can be considered as a unit containing a write IO request. The disk flushing on the disk flushing unit can be understood as writing the cache data corresponding to the write IO request into the persistent storage medium (such as a disk).

[0090] S211, when the cache control module finds that all the disk flushing units corresponding to the write IO requests in any used log object have been flushed, the cache control module notifies the cache log module to delete the log object.

[0091] It should be noted that after the cache log module is notified to delete the log object, if the deletion is not successful, the cache log module will be notified to delete the log object for multiple times until the deletion is successful.

[0092] S212, when the cache control module finds that all the disk flushing units corresponding to the target log object have been flushed, the cache control module switches the cache state to the disk flushing completion state.

[0093] S213, the cache control module notifies the data processing service module that the disk flushing is completed.

[0094] S214, the data processing service module reports the disk flushing completion event to the main monitoring service module.

[0095] S215, after receiving the disk flushing completion event reported by all the specified storage nodes, the main monitoring service module sends the new virtual node mapping table to the data processing service module on each storage node.

[0096] S216, the data processing service module updates the historical virtual node mapping table as the received new virtual node mapping table, and notifies the cache control module that the virtual node mapping table has been updated.

[0097] S217, the cache control module switches the cache state to the default state.

[0098] The above describes the data processing interaction process provided by the embodiment. The following further describes the data processing interaction process by combining the implementation schematic diagram of the write IO request cache processing:

[0099] First, as shown in Figure 3 , it is assumed that the IO requests counted by the first IO count are three write IO requests; at the current time, it is in the state of waiting for the first IO count to be cleared, the first IO count is 3, and the second IO count is 0. Among them, the write IO request 20 has entered the processing chain table of the cache log module, the write IO request 21 and the write IO request 22 have entered the processing chain table of the cache IO processing module, and at this time, the write IO request is to be written into the log object 11 (i.e. the currently used log object). Since no write IO request is completed at this time, the disk flushing chain table in the cache disk flushing module is empty (i.e. NULL). Here, when the write IO request is inserted into the disk flushing chain table, the client response will be returned immediately to indicate that the write IO request is completed, and at this time, the write IO request is completed.

[0100] At the next time, as shown in Figure 4 , the write IO request 20 to the write IO request 22 all enter the processing chain table of the cache log module, among which the write IO request 20 and 22 have been written into the log object 11, at this time, the log object 11 is full, and the write IO request 22 is about to be written into the log object 12, i.e. the currently used log object becomes the log object 12.

[0101] Then, after the first IO count is cleared, the cache IO processing module will notify the cache control module; the cache control module obtains the log object currently used by the cache log module as the target log object, and requires the cache log module to forcibly replace the currently used log object with the next unused log object. That is, as shown in Figure 5As shown, the currently used log object is switched from log object 12 to log object 13. After the log object switch is complete, the cache control module switches the cache state to the waiting state for the second IO count to be cleared. In this waiting state, each received write IO request needs to be judged. If the virtual node to which the object being operated belongs is in the virtual node migration subset, the write IO request is inserted into the IO cache queue and passed to the write-time redirection module for processing after the disk flush is completed. Otherwise, the write IO request is processed normally, and the write IO request count is added to the first IO count (i.e., the first IO count is reused) to prevent the second IO count from failing to clear.

[0102] Then as Figure 5 As shown, write IO requests 20, 21, and 22 return successful responses from the client, indicating that write IO requests 20, 21, and 22 have been processed, and the first IO count is equal to 0. While waiting for the first IO count to reset, write IO requests 23 and 24 arrive at the cache log module. These write IO requests 23 and 24 are still mapped to log object 12, and the currently used log object is log object 12, which is used as the target log object. After the cache log module changes the log object, the currently used log object switches to log object 13, thus ensuring that subsequent write IO requests will not write to log object 12. At this moment, write IO requests 20, 21, and 22 in the cache flushing module's flushing list have not completed flushing, therefore log object 11 is not deleted.

[0103] After the second IO count is reset to zero, the cache control module modifies the log object ID of the last flush unit in the cache flush module's flush list using the previously obtained target log object ID. A non-zero number under the flush unit indicates that the current log object is full, and subsequent write IO requests will be written to the new log object. See, for example... Figure 6 As shown, write IO requests 23 and 24 returned successfully, at which point the second IO count equaled 0. While waiting for the second IO count to reset, write IO requests 25, 26, and 27 arrived at the cache log module. Because the cache log module changed the log object, write IO requests 25, 26, and 27 were mapped to log object 13. Simultaneously, the log object ID of the last flush unit (i.e., flush unit 24) in the cache flush module's flush list was modified to 12.

[0104] Subsequently, the cache control module modifies the cache flushing module's flushing mode to forced flushing and waits for the flushing process to complete. See, for example... Figure 7 As shown, the condition for successful disk flushing is that the disk flushing unit with log object ID 12 (i.e., disk flushing unit 24) successfully flushes the disk.

[0105] The cache control module deletes the log object in a synchronous deletion manner. If the deletion is not successful, the deletion will be retried until the deletion is successful. For example, Figure 8 After the log object 12 is deleted, the cache control module sets the cache state to a disk flushing completion state. In the disk flushing completion state, the cache control module still needs to judge each received write IO request. If the corresponding virtual node of the write IO request is in the virtual node migration subset, the write IO request is transmitted to the write-time redirection module for processing. Otherwise, the write IO request is processed normally.

[0106] It should be noted that, as shown in Figure 8 After the log object 12 containing the last write IO request 22 corresponding to the suspected migration virtual node is deleted, the subsequent log objects no longer contain data of the migration virtual node. Therefore, after the log object 12 is deleted, it can be indicated that the cache data corresponding to the migration virtual node has been flushed, and at this time, the disk flushing can be stopped.

[0107] Finally, after the cache flushing module stops flushing, the cache control module notifies the data processing service module that the disk flushing is completed. The data processing service module notifies the main monitoring service module that the disk flushing is completed. When the main monitoring service module finds that all specified storage nodes have completed the disk flushing, the main monitoring service module sends a new virtual node mapping table to the data processing service of each storage node. The data processing service module receiving the new virtual node mapping table notifies the cache control module that the virtual node mapping table has changed, to instruct the cache control module to reset the cache state to a default state, and the cache processing is completed.

[0108] In the cache flushing, the write IO request corresponding to the non-migration virtual node can continue to be processed normally in the cache, to reduce the time of IO zero dropping. Only the write IO request corresponding to the migration virtual node needs to be transmitted to the write-time redirection module for processing. Compared with the full transmission mode, the influence on the performance of the distributed storage system can be effectively reduced. When the expansion state is entered, the write IO request of the virtual node to be migrated is transmitted to the write-time redirection module for processing, and no new log is generated in the cache. Therefore, as long as the log object corresponding to the in-flight IO is deleted, it can be ensured that there is no log object of the migration virtual node in the original physical node after the expansion is completed.

[0109] Thus, the method provided in the embodiment of the present application is described, and the device provided in the embodiment of the present application is described as follows. Thus, the method provided in the embodiment of the present application is described, and the device provided in the embodiment of the present application is described as follows.

[0110] As an embodiment, the embodiment also provides a data processing apparatus. Referring to Figure 9 , Figure 9 The apparatus structure diagram provided by the embodiment of the present application is shown in FIG. 9. As shown in the figure, the apparatus 900 comprises: Figure 9

[0111] The obtaining module 901 is configured to obtain a virtual node migration set, determine a virtual node migration subset corresponding to the current node according to the virtual node migration set, and record information of at least one data processing request received before the current time point and not processed into a first processing chain list; the virtual node migration set is composed of at least one virtual node mapped to a newly added storage node during the current node expansion.

[0112] The receiving module 902 is configured to receive a data processing request, when the data processing request is a write data request, the write data request indicating a node identifier corresponding to a virtual node matching a storage location to which data is to be written; if it is determined that the virtual node corresponding to the node identifier is in the virtual node migration subset, adding the write data request to a cache queue for caching write data requests requiring write-time redirection processing; otherwise, adding information of the write data request to a second processing chain list until it is found that all data processing requests in the first processing chain list are processed.

[0113] The disk flushing module 903 is configured to, when it is found that all write data requests in the second processing chain list are processed, flush cache data corresponding to the write data requests in the current disk flushing chain list until a set disk flushing stop condition is reached; the set disk flushing stop condition refers to that a target log object is deleted or the target log object records an operation log corresponding to the last processed write data request in the first processing chain list.

[0114] As an embodiment, the virtual node migration set is determined by a target storage node based on all virtual nodes preconfigured and all storage nodes contained in the distributed storage system after the current node expansion; the target storage node refers to a specified storage node detecting a node expansion event.

[0115] As an embodiment, the current node is preconfigured with N threads for processing data processing requests, N being greater than 1; each thread corresponds to a partial virtual node migration subset; the partial virtual node migration subsets corresponding to the threads are different; the partial virtual node migration subsets corresponding to the threads constitute the virtual node migration subset.

[0116] As an embodiment, the target log object is determined by the following way: when it is found that all data processing requests in the first processing chain list are processed, obtaining a currently used log object from a plurality of log objects configured to record operation logs corresponding to write data requests to serve as the target log object. ​

[0117] As an embodiment, after obtaining the currently used log object as the target log object, the apparatus further comprises a determining module configured to determine the next unused log object in the configured log objects as the currently used log object.

[0118] As an embodiment, the disk flushing module 903 is further configured to, for each write data request in the current disk flushing link table, after completing disk flushing of the cache data corresponding to the write data request, check whether the log object recording the operation log corresponding to the write data request still contains the operation log corresponding to the write data request that has not completed disk flushing, and if not, delete the log object recording the operation log corresponding to the write data request.

[0119] As an embodiment, the disk flushing module 903 is further configured to, when it is found that the set disk flushing stop condition is reached, determine that the cache data corresponding to each virtual node in the virtual node migration subset has completed disk flushing, and report a disk flushing completion event indicating that the cache data has completed disk flushing to the target storage node.

[0120] The target storage node, after receiving all the disk flushing completion events reported by the specified storage nodes, sends a virtual node mapping table update event corresponding to each storage node included in the distributed storage system after the current node expansion to the storage node, to instruct the storage node to update the local historical virtual node mapping table to a new virtual node mapping table indicated by the virtual node mapping table update event. The new virtual node mapping table is determined according to all the virtual nodes preconfigured and all the storage nodes included in the distributed storage system after the current node expansion, and thus, the cache data corresponding to each virtual node in the virtual node mapping table update event has completed disk flushing. Figure 9 The structure of the apparatus is described.

[0121] The implementation process of the functions and roles of each unit in the above apparatus is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0122] For the device embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The device embodiments described above are only illustrative, and the modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, i.e. they can be located in one place or distributed on multiple network modules. According to actual needs, part or all of the modules can be selected to achieve the purpose of the present application. Those skilled in the art can understand and implement without creative labor.

[0123] Please refer to Figure 10A hardware structure schematic diagram of an electronic device is provided for an exemplary embodiment of the present application. The electronic device can include a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004. The processor 1001, the communication interface 1002 and the memory 1003 complete the communication among each other through the communication bus 1004. Among them, the memory 1003 stores computer program instructions; the processor 1001 can execute the steps of the method described in the above embodiment by executing the computer program instructions stored in the memory 1003. The electronic device can also include other hardware according to the actual function of the electronic device, which will not be described here.

[0124] Correspondingly, the present application also provides a computer readable storage medium, which stores a plurality of computer program instructions. When the computer program instructions are executed by a processor, the method disclosed in the above exemplary embodiments of the present application can be implemented.

[0125] Exemplarily, the above computer readable storage medium can be any electronic, magnetic, optical or other physical storage apparatus, which can contain or store information such as executable instructions, data, etc. For example, the computer readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state disk, any type of storage disk (such as optical disk, dvd, etc.), or similar storage medium, or combination thereof. The processor and the memory can be supplemented by or incorporated into a special logic circuit.

[0126] The above is only a preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A data processing method, characterized by, The method is applied to any specified storage node in a distributed storage system; the specified storage node refers to a storage node included in the distributed storage system before the current node expansion; the method comprises: obtaining a virtual node migration set, determining a virtual node migration subset corresponding to the current node according to the virtual node migration set, and recording information of at least one data processing request received before a current time point and not processed into a first processing chain table; the virtual node migration set is composed of at least one virtual node mapped to a newly added storage node during the current node expansion; receiving a data processing request, when the data processing request is a write data request, the write data request indicating a node identifier corresponding to a virtual node matching a storage location to which data is to be written; if it is determined that the virtual node corresponding to the node identifier is in the virtual node migration subset, the write data request is added to a cache queue for caching write data requests requiring write-time redirection processing; otherwise, information of the write data request is added to a second processing chain table until it is found that all data processing requests in the first processing chain table are processed. When it is found that all write data requests in the second processing chain table are processed, cached data corresponding to the write data requests in a current disk flushing chain table are flushed until a set disk flushing stop condition is reached; the set disk flushing stop condition refers to that a target log object is deleted; the target log object records an operation log corresponding to a last processed write data request in the first processing chain table.

2. The method of claim 1, wherein, The virtual node migration set is determined by a target storage node based on all virtual nodes pre-configured and all storage nodes included in the distributed storage system after the current node expansion; The target storage node refers to a specified storage node detecting a node expansion event.

3. The method of claim 1, wherein, The current node pre-configures N threads for processing data processing requests, N being greater than 1; each thread corresponds to a partial virtual node migration subset; the partial virtual node migration subsets corresponding to the threads are different; The partial virtual node migration subsets corresponding to the threads constitute the virtual node migration subset.

4. The method of claim 1, wherein, The target log object is determined in the following way: When it is found that all data processing requests in the first processing chain table are processed, a currently used log object is obtained from a plurality of configured log objects for recording operation logs corresponding to write data requests to serve as the target log object.

5. The method of claim 4, wherein, After obtaining the currently used log object to serve as the target log object, the method further comprises: the next unused log object in the configured log objects is taken as the currently used log object.

6. The method of claim 1, wherein, The method further comprises: for each write data request in the current disk flushing chain table, after flushing cached data corresponding to the write data request, it is checked whether the log object recording an operation log corresponding to the write data request still contains an operation log corresponding to a write data request not completed flushing; if not, the log object recording the operation log corresponding to the write data request is deleted.

7. The method of claim 2, wherein, The method further comprises: When it is found that the set brush disc stop condition is reached, it is determined that the cache data corresponding to each virtual node in the virtual node migration subset is completed, and a brush disc completion event indicating that the cache data is completed is reported to the target storage node; The target storage node, after receiving the brush disc completion events reported by all specified storage nodes, sends a virtual node mapping table update event corresponding to each storage node included in the distributed storage system after this node expansion to the storage node, to instruct the storage node to update the local historical virtual node mapping table to a new virtual node mapping table indicated by the virtual node mapping table update event; the new virtual node mapping table is determined according to all virtual nodes preconfigured and all storage nodes included in the distributed storage system after this node expansion.

8. A data processing apparatus, characterized by, The device is configured in any specified storage node in the distributed storage system; the specified storage node refers to a storage node included in the distributed storage system before this node expansion; the device comprises: An obtaining module is configured to obtain a virtual node migration set, determine a virtual node migration subset corresponding to the current node according to the virtual node migration set, and record information of at least one data processing request received before the current time point and not processed into a first processing chain table; the virtual node migration set is composed of at least one virtual node mapped to a newly added storage node during this node expansion; A receiving module is configured to receive a data processing request; when the data processing request is a write data request, the write data request indicates a node identifier corresponding to a virtual node matching a storage location to which data is to be written; if it is determined that the virtual node corresponding to the node identifier is in the virtual node migration subset, the write data request is added to a cache queue for caching write data requests requiring write redirection processing; otherwise, information of the write data request is added to a second processing chain table until it is found that all data processing requests in the first processing chain table are processed. A brush disc module is configured to perform brush disc on cache data corresponding to a write data request in a current brush disc chain table when it is found that all write data requests in the second processing chain table are processed, until a set brush disc stop condition is reached; the set brush disc stop condition refers to that a target log object is deleted; the target log object records an operation log corresponding to the last processed write data request in the first processing chain table.

9. The apparatus of claim 8, wherein, The virtual node migration set is determined by the target storage node based on all virtual nodes preconfigured and all storage nodes included in the distributed storage system after this node expansion; The target storage node refers to a specified storage node detecting a node expansion event; And / or, The current node is preconfigured with N threads for processing data processing requests, N being greater than 1; each thread corresponds to a partial virtual node migration subset; The partial virtual node migration subsets corresponding to each thread are different; the partial virtual node migration subsets corresponding to each thread constitute the virtual node migration subset; And / or, The target log object is determined by the following way: when it is found that all data processing requests in the first processing chain table are processed, the currently used log object is obtained from the configured multiple log objects for recording operation logs corresponding to write data requests as the target log object; and / or, After obtaining the currently used log object as the target log object, the apparatus further comprises a determining module for determining the next unused log object in the configured log objects as the currently used log object; and / or, The disk flushing module is further configured to: for each write data request in the current disk flushing chain table, after completing disk flushing of the cache data corresponding to the write data request, check whether the log object recording the operation log corresponding to the write data request still contains the operation log corresponding to the write data request with unfinished disk flushing; if not, delete the log object recording the operation log corresponding to the write data request; and / or, The disk flushing module is further configured to: when it is found that the set disk flushing stop condition is reached, determine that the cache data corresponding to each virtual node in the virtual node migration subset has completed disk flushing, and report a disk flushing completion event indicating that the cache data has completed disk flushing to the target storage node; wherein the target storage node, after receiving all disk flushing completion events reported by the specified storage nodes, sends a virtual node mapping table update event corresponding to each storage node included in the distributed storage system after this node expansion to the storage node, to instruct the storage node to update the local historical virtual node mapping table to a new virtual node mapping table indicated by the virtual node mapping table update event; the new virtual node mapping table is determined according to all virtual nodes and all storage nodes included in the distributed storage system after this node expansion.

10. An electronic device, comprising: The electronic device includes: a processor; and a memory having computer program instructions stored therein, the computer program instructions, when executed by the processor, causing the processor to perform the steps in any one of claims 1 to 7.

Images