Distributed data storage control method and device, readable medium and electronic equipment

Abstract

The present disclosure relates to a distributed data storage control method and device, readable medium and electronic equipment, wherein the method comprises: after a worker node receives a write lock and modifies a file size according to the write lock, the modified file size is first cached in the worker node; and when a write lock request for recovering the write lock sent by a distributed lock management and control node is responded, the modified file size is transmitted to the distributed lock management and control node, so that the distributed lock management and control node updates the file size. Thus, in a high-concurrency file size modification scenario, the request quantity of synchronizing the file size to the distributed lock management and control node can be avoided from being too large, and the performance and concurrency of the distributed lock management and control node are affected.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular, to a distributed data storage control method and device, readable medium and electronic equipment. BACKGROUND

[0002] With the rapid development of computer technology, distributed systems have been widely applied. In the related technology, after the working node in the distributed system modifies the obtained shared resource, the modified shared resource needs to be transmitted to the management node of the shared resource in real time to ensure the consistency of the shared resource. SUMMARY

[0003] This summary is provided to introduce a selection of concepts that are further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0004] In a first aspect, the present disclosure provides a distributed data storage control method, comprising:

[0005] In response to a first modification request for a target file sent by a client, a first acquisition request for acquiring a first write lock, a first inode identifier and a first file size is sent to a distributed lock management node, the first write lock is used to control the write operation of the node to the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file;

[0006] The first write lock, the first file size and the first inode identifier sent by the distributed lock management node in response to the first acquisition request are received, the first file size is modified according to the first modification information in the first modification request to obtain a second file size, and the second file size is cached to a target position in a first index data table according to the first inode identifier, the first index data table is stored in the current node;

[0007] In response to a recovery request for recovering the first write lock sent by the distributed lock management node, the first write lock, the first inode identifier and the second file size are sent to the distributed lock management node, so that the distributed lock management node updates the first file size corresponding to the first inode identifier in a second index data table to the second file size, and the second index data table is stored in the distributed lock management node.

[0008] In a second aspect, the present disclosure provides a distributed data storage control method, comprising:

[0009] In response to a first acquisition request sent by a node for acquiring a first write lock, a first inode identifier, and a first file size, the first write lock, the first file size, and the first inode identifier are sent to the node. The first write lock is used to control the node's write operations on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file.

[0010] In response to the first acquisition request sent by the other nodes, a revocation request for revoking the first write lock is sent to the node;

[0011] The node receives the first write lock, the first index node identifier, and the second file size sent in response to the revocation request. Based on the first index node identifier, the node obtains the first file size in the second index data table and updates the first file size in the second index data table to the second file size. The second index data table is stored on the current node. The second file size is obtained by the node modifying the first file size sent by the distributed lock control node according to the first modification information in the first modification request. The first modification request is sent by the client.

[0012] Thirdly, this disclosure provides a distributed data storage control device, the device comprising:

[0013] The first sending module is used to respond to the first modification request for the target file sent by the client, and send a first acquisition request to the distributed lock control node to acquire a first write lock, a first inode identifier, and a first file size. The first write lock is used to control the write operation on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file.

[0014] The first receiving module is configured to receive the first write lock, the first file size, and the first index node identifier sent by the distributed lock control node in response to the first acquisition request, modify the first file size according to the first modification information in the first modification request to obtain the second file size, and cache the second file size in the target position of the first index data table according to the first index node identifier, wherein the first index data table is stored in the current node;

[0015] The second sending module is configured to respond to a revocation request sent by the distributed lock management node for revoking the first write lock, send the first write lock, the first index node identifier, and the second file size to the distributed lock management node, so that the distributed lock management node updates the first file size corresponding to the first index node identifier in the second index data table to the second file size, and the second index data table is stored in the distributed lock management node.

[0016] Fourthly, this disclosure provides a distributed data storage control device, the device comprising:

[0017] The eighth sending module is used to respond to a first acquisition request sent by a node for acquiring a first write lock, a first inode identifier, and a first file size, and send the first write lock, the first file size, and the first inode identifier to the node. The first write lock is used to control the node's write operation on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file.

[0018] The ninth sending module is used to send a revocation request to the node for revoking the first write lock in response to the first acquisition request sent by the other nodes.

[0019] The update module is used to receive the first write lock, the first index node identifier, and the second file size sent by the node in response to the revocation request; obtain the first file size in the second index data table according to the first index node identifier; and update the first file size in the second index data table to the second file size. The second index data table is stored on the current node. The second file size is obtained by the node modifying the first file size sent by the distributed lock control node according to the first modification information in the first modification request. The first modification request is sent by the client.

[0020] Fifthly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in any one of the first or second aspects.

[0021] Sixthly, this disclosure provides an electronic device, comprising:

[0022] A storage device on which computer programs are stored;

[0023] A processing device for executing the computer program in the storage device to implement the steps of the method in either the first or second aspect.

[0024] Through the above technical solution, after receiving the write lock, inode identifier, and file size from the distributed lock management node, the worker node can modify the file size according to the modification information in the modification request, and cache the modified file size in the target location corresponding to the inode identifier in the first index data table. When responding to a write lock revocation request, the worker node sends the inode identifier and the file size corresponding to that inode identifier in the first index data table to the distributed lock management node, so that the distributed lock management node updates the file size corresponding to that inode identifier in the second index data table to the received file size. Because the worker node does not immediately send the modified file size to the distributed lock management node after modification, but instead caches the modified file size in the first index data table and transmits it to the distributed lock management node when responding to a revocation request, this avoids excessive requests to synchronize file sizes to the distributed lock management node in high-concurrency file size modification scenarios, thus preventing the performance and concurrency of the distributed lock management node from being affected.

[0025] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0026] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings:

[0027] Figure 1 This is a schematic diagram of a network memory architecture according to an exemplary embodiment of the present disclosure;

[0028] Figure 2 This is a flowchart illustrating a distributed data storage control method according to an exemplary embodiment of the present disclosure;

[0029] Figure 3 This is a schematic diagram illustrating an inconsistency in file size between the distributed lock control node and the resource management node caused by a file size update, according to an exemplary embodiment of this disclosure.

[0030] Figure 4 This is a flowchart illustrating an exemplary embodiment of the present disclosure for obtaining file size;

[0031] Figure 5 This is a flowchart illustrating another distributed data storage control method according to an exemplary embodiment of the present disclosure;

[0032] Figure 6This is a flowchart illustrating an update of file size within an update cycle according to an exemplary embodiment of the present disclosure;

[0033] Figure 7 This is a flowchart illustrating a file size check according to an exemplary embodiment of the present disclosure;

[0034] Figure 8 This is a flowchart illustrating a file size update according to an exemplary embodiment of the present disclosure;

[0035] Figure 9 This is a structural block diagram of a distributed data storage control device according to an exemplary embodiment of the present disclosure;

[0036] Figure 10 This is a structural block diagram of another distributed data storage control device according to an exemplary embodiment of the present disclosure;

[0037] Figure 11 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0038] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0039] As mentioned in the background section, with the rapid development of computer technology, distributed systems have been widely used. To prevent interference between multiple worker nodes in a distributed system, distributed locks (write locks or read locks) are used to schedule worker nodes. That is, when a worker node is granted a distributed lock, other worker nodes cannot invoke that distributed lock.

[0040] In related technologies, after a worker node modifies the acquired shared resources, it needs to transmit the modified shared resources to the management node of the shared resources in real time to ensure the consistency of the shared resources.

[0041] For example, such as Figure 1 As shown, the storage architecture of a network storage system includes a proxy server, a block server (BS), and a directory tree server.

[0042] The proxy is responsible for parsing protocol requests sent by clients, including network file system, server message blocks, and user-space file system requests, and then forwarding the parsed requests to the backend storage cluster. During this process, the proxy needs to interact with the directory tree server and the browser / storage unit (BS).

[0043] The directory tree server is used to manage the file directory tree, and is responsible for metadata services and distributed lock services, managing all metadata attributes of files.

[0044] The Base Server (BS) is responsible for file read operations, write operations, and data persistence. In other words, the proxy needs to interact with the BS for both file read and write operations. Furthermore, the BS manages segment units; a file is split into 16 segments of 64KB each. When the proxy issues an I / O request, it needs to calculate the corresponding segment based on the modification information in the request, namely the offset and length, and then send the I / O request to its BS.

[0045] In related technologies, the storage architecture incorporates a distributed lock for metadata and optimizes for file size. Specifically, the size is moved from the original browser-server (BS) storage to a directory tree server. This ensures that when a proxy needs to retrieve the size, it obtains it directly from the directory tree server instead of directly from the BS. A distributed read-write lock is also designed for the size, ensuring read-read compatibility and read-write and write-write mutual exclusion. With the distributed lock available, the proxy can directly trust the cached size without needing to retrieve it from the backend cluster. Without the distributed lock, the proxy needs to request the distributed lock from the directory tree server, record the lock upon successful request, and cache the size.

[0046] When a proxy requests a size distributed lock from the directory tree server, it needs to differentiate between a size write lock and a size read lock based on the use case. For example, a request to retrieve an attribute (GetAttr) only requires a size read lock, while an append write request requires a size write lock. Upon receiving a size distributed lock request, the directory tree server first revokes conflicting distributed locks based on read-write lock conflicts before allocating a new one. Specifically, considering read-write concurrency optimization, the size read lock is optimized as a non-essential lock. When the directory tree server receives a size read lock request, if it finds that a size write lock is already allocated and the size has recently been updated, it will not revoke the size write lock, nor will it allocate a size read lock; instead, it will only return a temporary size for the proxy to use.

[0047] In scenarios where the proxy modifies the size (i.e., append and truncation operations), the proxy needs to first acquire a size write lock and protect it from revocation using a memory lock. Then, while the size write lock is active, the proxy writes the book (BS). After writing the BS, the modified size is synchronously flushed to the directory tree server before a success message is returned to the user.

[0048] However, in the current architecture's size processing logic, after the proxy modifies the size, it immediately synchronizes the modified size to the directory tree server. This results in a large number of requests to synchronize the size to the directory tree server in high-concurrency size modification scenarios, leading to excessive utilization of the directory tree server's central processing unit and affecting the performance and concurrency of the distributed lock control node.

[0049] In view of this, the present disclosure provides a distributed data storage control method, apparatus, readable medium, and electronic device to solve the above-mentioned technical problems.

[0050] The embodiments of this disclosure will be further explained below with reference to the accompanying drawings.

[0051] Figure 2 This is a flowchart illustrating a distributed data storage control method according to an exemplary embodiment of the present disclosure, with reference to... Figure 2 The method may include the following steps:

[0052] S201: In response to the first modification request for the target file sent by the client, a first acquisition request is sent to the distributed lock control node to acquire a first write lock, a first inode identifier, and a first file size. The first write lock is used to control the write operation of the worker node on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file.

[0053] It should be understood that a distributed lock management node refers to a node used to manage or schedule distributed locks, while a worker node refers to a node that needs to acquire a distributed lock to obtain a target resource. Depending on the application scenario, the specific distributed lock management node and worker nodes can be determined according to the actual situation. For example, in the storage architecture of a network storage device, the distributed lock management node can be a directory tree server, and the worker node can be a proxy. The target resource refers to the content that the client wants to read or write, such as a file, video, audio, etc. The specific form of the target resource depends on the actual application, and this disclosure does not impose any limitations on it.

[0054] It's also important to understand that an inode is a data structure used to store file metadata, and each file has a unique inode in the file system. Through inodes, the file system can quickly access file attributes and content without searching by filename. Metadata refers to data describing a file or directory, including file attribute information such as filename, file size, creation time, and modification time. To quickly locate the inode corresponding to each file, each inode is typically assigned an inode identifier, which is then bound to the corresponding file and inode. Therefore, when retrieving the file attributes of a target file, the process involves first obtaining the target inode identifier based on the target file, the binding relationship between the file and the inode identifier, then obtaining the target inode based on the target inode identifier and the binding relationship between the inode identifier and the inode, and finally, accessing the target inode to obtain the file attributes of the target file.

[0055] The inode identifier can be set according to the actual situation. This disclosure does not impose any restrictions on it. For example, 1111 can be used to represent inode 1, 2222 can be used to represent inode 2, or AAA can be used to represent inode 1, and BBB can be used to represent inode 2, etc.

[0056] S202: Receive the first write lock, the first file size, and the first index node identifier sent by the distributed lock control node in response to the first acquisition request; modify the first file size according to the first modification information in the first modification request to obtain the second file size; and cache the second file size in the target position of the first index data table according to the first index node identifier, wherein the first index data table is stored in the current working node.

[0057] The first modification information may include offset and length, etc., so that the starting position of the data to be modified in the target file can be determined according to the offset, and the length of the data to be modified can be determined according to the length. Thus, the second file size can be obtained according to the received first file size, offset and length.

[0058] The first index data table refers to the data table in the current working node used to store the index data of files. The index data may include inode identifiers and inodes, etc. The structure of the first index data table can be set according to actual conditions, and this embodiment does not impose any restrictions on it. For example, the data structure of the first index data table can be as shown in Table 1.

[0059] Table 1 First Index Data Table

[0060]

[0061] S203: In response to the revocation request sent by the distributed lock management node for revoking the first write lock, the first write lock, the first index node identifier, and the second file size are sent to the distributed lock management node, so that the distributed lock management node updates the first file size corresponding to the first index node identifier in the second index data table to the second file size, and the second index data table is stored in the distributed lock management node.

[0062] The second index data table refers to the data table in the distributed lock control node used to store file index data. The index data may include inode identifiers and inodes, etc. The structure of the second index data table may be the same as or different from that of the first index data table; this disclosure does not impose any restrictions on this.

[0063] Through the above technical solution, after receiving the write lock, inode identifier, and file size from the distributed lock management node, the worker node can modify the file size according to the modification information in the modification request, and cache the modified file size in the target location corresponding to the inode identifier in the first index data table. When responding to a write lock revocation request, the worker node sends the inode identifier and the file size corresponding to that inode identifier in the first index data table to the distributed lock management node, so that the distributed lock management node updates the file size corresponding to that inode identifier in the second index data table to the received file size. Because the worker node does not immediately send the modified file size to the distributed lock management node after modification, but instead caches the modified file size in the first index data table and transmits it to the distributed lock management node when responding to a revocation request, this avoids excessive requests to synchronize file sizes to the distributed lock management node in high-concurrency file size modification scenarios, thus preventing the performance and concurrency of the distributed lock management node from being affected.

[0064] In possible implementations, the method may further include:

[0065] In response to the second modification request for the target file sent by the client, the size of the second file is modified according to the second modification information in the second modification request to obtain the modified second file size, and the second file size in the first index data table is updated to the modified second file size.

[0066] It should be understood that a worker node can change the file size multiple times while holding a write lock.

[0067] Through the above technical solution, worker nodes do not need to transmit the modified file size to the distributed lock management node every time. This reduces the number of communications between the distributed lock management node and worker nodes, as well as the number of times the distributed lock management node updates the file size in the second index data table, thereby improving the performance and concurrency of the distributed lock management node. On the other hand, since worker nodes cache the modified file size after modifying it, they do not need to retrieve it from the distributed lock management node the next time they read or modify the file size, thus improving the efficiency of file size retrieval.

[0068] As mentioned earlier, during the write lock holding period, the modified file size is not immediately transmitted to the distributed lock management node. Instead, it is transmitted only when the write lock is revoked. When the write lock holding time is long, the distributed lock management node may not obtain the latest file size for an extended period. Consequently, when calculating the quota, the distributed lock management node will still use the file size before the update, leading to errors in the calculated quota. To overcome this technical problem, this embodiment periodically transmits the file size back to the distributed management system, allowing the distributed management system to update the file size in the second index data table in a timely manner.

[0069] That is, in possible implementations, the method may further include:

[0070] The system periodically acquires write lock validity information to characterize whether the first write lock is valid. When the write lock validity information indicates that the first write lock is valid, it sends the first inode identifier, the second file size, and the first write lock identifier to the distributed lock management node. This allows the distributed lock management node to retrieve the first file size and the second write lock identifier from the second index data table based on the first inode identifier, and update the first file size to the second file size based on the first write lock identifier and the second write lock identifier. The first write lock identifier is used to identify the first write lock, and it is sent by the distributed lock management node when sending the first write lock to the worker node. The second write lock identifier is used to identify the second write lock currently controlling the write operation on the target file.

[0071] The period for obtaining write lock validity information can be 10s, 30s or 1min, and this embodiment does not impose any restrictions on this.

[0072] The write lock validity information can be obtained from the first index data table. For example, when a worker node receives the first write lock, the first file size, and the first index node identifier from the distributed lock management node, it first retrieves the corresponding index node from the first index data table based on the first index node identifier, and then records the write lock validity information in the index node. For example, it might enter "received the first write lock at time t" or "true" in the write lock validity information field. Similarly, when a worker node receives a revocation request from the distributed lock management node, it can also first retrieve the corresponding index node from the first index data table based on the first index node identifier, and then record the write lock validity information in the index node. For example, it might enter "revoked the first write lock at time t+10" or "false" in the write lock validity information field.

[0073] When the write lock validity information indicates that the first write lock is invalid, it means that the first write lock has been revoked, so it does not need to be updated.

[0074] The write lock identifier can be set according to actual conditions, and this embodiment does not impose any restrictions on it. For example, the letter A can be used to represent write lock 1, the letter B to represent write lock 2, and the letter C to represent write lock 3. Alternatively, the number 1 can be used to represent write lock 1, the number 2 to represent write lock 2, and the number 3 to represent write lock 3, etc.

[0075] It should be understood that write locks and write lock identifiers generally have a one-to-one correspondence. By sending the write lock identifier to the distributed lock management node when updating the file size, update errors can be avoided.

[0076] For example, suppose worker node 1 sends a second file with a size of 10 during a certain sending cycle. Before the distributed lock management node receives the second file size sent by worker node 1, worker node 1 updates the second file size to 20. Simultaneously, the distributed lock management node reclaims write lock 1 from worker node 1. Since the second file size in worker node 1 is 20 when write lock 1 is reclaimed, the distributed lock management node updates the first file size in the second index data table to 20. At the same time, the distributed lock management node also allocates write lock 2, which controls worker nodes' write operations on the target file, to worker node 2. If the distributed lock management node only receives the second file size previously sent by worker node 1 at this point, without sending a write lock identifier, the distributed lock management node will change the first file size in the second index data table from 20 to 10, resulting in an update error.

[0077] In a possible implementation, to avoid updating file sizes that haven't changed between two update intervals back to the distributed lock management node, an update flag can be added to the index node where the file size has been updated. For example, a "dirty" flag can be added to the target index node where the file size has changed but hasn't yet been updated back to the distributed lock management node. This allows the "dirty" flag to determine whether the file size needs to be updated back to the distributed lock management node during an update, avoiding invalid updates and wasted resources. Correspondingly, the distributed lock management node needs to clear the corresponding "dirty" flag when updating the file size in the second index data table.

[0078] It should be understood that a file can only be modified by one worker node at a time, but a worker node can modify multiple files simultaneously. Therefore, a worker node can contain multiple inode identifiers, multiple write lock identifiers, and multiple file sizes. Thus, in a possible implementation, when a worker node contains multiple file sizes, multiple inode identifiers, and multiple write lock identifiers, the file sizes of all files in the worker node can be updated to the second index data table during each update cycle.

[0079] For example, update requests that include multiple target inode identifiers, multiple file sizes, and multiple write lock identifiers can be generated periodically and sent to the distributed lock management node, so that the distributed lock management node updates the file size in the second index data table according to the target inode identifiers, file sizes, and write lock identifiers included in the update request.

[0080] The number of inode identifiers, file sizes, or write lock identifiers in the update request can be set according to actual conditions, and this embodiment does not impose any restrictions on this. Because the distributed lock control node, when updating the file size corresponding to the inode identifier, will acquire a memory lock on the inode in the second index data table that needs its file size updated during the update process to prevent worker nodes from changing the file size corresponding to the inode identifier during the update process. Therefore, to prevent the distributed lock control node from occupying the inode in the second index data table that needs its file size updated for an extended period, affecting the worker nodes' read and write operations on the inode, the number of inode identifiers, file sizes, or write lock identifiers in the update request cannot be set too large.

[0081] The above technical solution can pack multiple index node identifiers, multiple file sizes, and multiple write lock identifiers into a single update request, thereby reducing the number of communications between worker nodes and distributed lock management nodes, reducing communication overhead, and further improving the performance of worker nodes and distributed lock management nodes.

[0082] In a possible implementation, to prevent the loss of write locks and modified file sizes due to worker node anomalies (such as failures or restarts) after modifying file sizes, thus ensuring that the file size corresponding to the inode identifier in the second index data table cannot be updated and consequently does not accurately reflect the file size of the file corresponding to the inode identifier, this embodiment proposes a three-level storage architecture. In addition to storing the index data of each file on the worker nodes and the distributed lock control node, the index data of each file is also stored on the resource management node. Therefore, when the file size is lost in the worker nodes, the file size corresponding to the inode identifier can be retrieved from the resource management node based on the inode identifier.

[0083] The file size in the resource management node can be determined and saved before the worker node updates the file size in the first index data table. For example, before updating the file size in the first index data table based on the modification information, the worker node first modifies the file corresponding to the index data identifier in the resource management node according to the modification information and the index data identifier to obtain the modified file size. This allows the worker node to retrieve the corresponding file size from the resource management node based on the index node identifier after a failure.

[0084] In this context, a resource management node refers to a node used for storing and managing files, and its specific location can be determined based on the actual situation. For example, in a network storage architecture, the resource management node can be a browser (BS).

[0085] Furthermore, since the distributed lock control node and worker nodes cannot detect whether the file size in the second index data table is abnormal, this embodiment also sets a node validity flag `trust_dn_size` for each index node identifier in the second index data table to indicate whether the file size corresponding to the index node identifier is valid. For example, when `trust_dn_size` is false, it indicates that the file size corresponding to the index node identifier is invalid or incorrect; when `trust_dn_size` is true, it indicates that the file size corresponding to the index node identifier is valid or correct. Therefore, the worker node can determine whether the file size sent by the distributed lock control node is correct based on `trust_dn_size`. If incorrect, it can obtain the file size from the resource management node.

[0086] That is, according to one embodiment of this disclosure, the method may further include:

[0087] Simultaneously with receiving the first write lock, the first file size, and the first index node identifier, the system receives a node validity flag sent by the distributed lock control node to indicate whether the first file size is valid. When the node validity flag indicates that the first file size is valid, the system caches the first file size in the target position of the first index data table according to the first index node identifier. When the node validity flag indicates that the file size is invalid, the system sends a read request to the resource management node to read the file size of the target file, so that the resource management node obtains the third file size of the target file from the third index data table according to the first index node identifier in the read request. The third index data table is stored on the resource management node. The system receives the third file size and caches it in the target position of the first index data table according to the index node identifier.

[0088] The third index data table refers to the data table in the resource management node used to store index data for files. The index data may include inode identifiers and inodes, etc. The structure of the third index data table may be the same as or different from the structures of the first and second index data tables; this disclosure does not impose any restrictions on this.

[0089] It's worth noting that the default value of `trust_dn_size` is `true`. When the distributed lock management node fails to reclaim a write lock, it changes the value of `trust_dn_size` from `true` to `false`. Failure to reclaim a write lock can include the loss of the worker node's write lock or the failure to receive a response within a preset time.

[0090] In addition, to ensure that the distributed lock management node can still know which write locks were lost after a restart, the distributed lock management node needs to persist the trust_dn_size.

[0091] Additionally, it's worth noting that when the write lock is lost and the file size is obtained from the resource management node using the above method, the distributed lock control node still cannot change `trust_dn_size` to `true` for a certain period (e.g., 60 seconds). That is, within 60 seconds, worker nodes need to obtain the file size from the third index table to avoid inconsistencies between the file size in the third index table and the file size in the second index table.

[0092] For example, such as Figure 3As shown, assuming file A's original size is 10, worker node 1 holds a write lock on file A and sends a write request (to modify file A) to the resource management node at 2.9 seconds. Worker node 1 fails at 3 seconds. The resource management node receives the write request and completes its processing at 12.9 seconds, modifying file A and changing its size in the third index data table to 20. However, at 4 seconds, the distributed lock control node receives a request from worker node 2 to acquire a write lock on file A. Due to worker node 1's failure, the distributed lock control node, after failing to reclaim the write lock from worker node 1, sets the `trust_dn_size` corresponding to file A to false and sends the write lock and `trust_dn_size` to worker node 2. Since `trust_dn_size` is false, worker node 2 retrieves the file size of file A from the third index data table, which is 10. Then, at 5 seconds, the distributed lock control node reclaims the write lock sent to worker node 2, receiving the file size of 10 returned by worker node 2. However, after the write request sent by worker node 1 is completed at the resource management node, the file size of file A in the third index data table is updated to 20. This results in a discrepancy between the file size of file A in the third index data table and the second index data table; the file size of file A in the second index data table is 10, while the file size of file A in the third index data table is 20.

[0093] The root cause of the inconsistent file sizes is that write requests were already being sent before the worker node failed. Therefore, the distributed lock management node needs a sufficiently long time before it can trust the received file size, allowing enough time for the write requests to complete. So, when worker node 2 sends the file size to the distributed lock management node, if the node finds that `trust_dn_size` has been set to false for less than 60 seconds, it will not accept the file size. Furthermore, to prevent worker node 2 from also failing within 60 seconds, and having also sent write requests before its failure, the 60-second timeframe needs to be extended if worker node 2 fails. For example, if worker node 2 fails after 40 seconds, the distributed lock management node will only accept the returned file size and set `trust_dn_size` to true after 100 seconds.

[0094] As can be seen from the above embodiments, the file size cached at the target location can be sent by the distributed management node, the resource management node, or modified according to a modification request. Therefore, the cache structure in the first index data table can be modified to include a first cache unit and a second cache unit. The first cache unit caches the file size sent by the distributed management node and the file size modified according to the modification request, while the second cache unit caches the file size sent by the resource management node. Thus, upon receiving the file size, it can be stored differently based on its source, improving the readability of the file size data.

[0095] For example, the cache structure in the first index data table can be set to {valid_size, size_from_bs}, where valid_size is used to cache the file size sent by the distributed management node and the file size modified according to the modification request, and size_from_bs is used to cache the file size sent by the resource management node. The initial values ​​of both valid_size and size_from_bs are -1. When a worker node requests a write lock, the distributed lock control node returns the write lock along with the node validity flag trust_dn_size. If trust_dn_size is true, the worker node sets valid_size to the file size sent by the distributed lock control node; if trust_dn_size is false, the worker node does not set valid_size, i.e., the valid_size value remains -1. Simultaneously, a read request and index node identifier are sent to the resource management node to obtain the file size from the resource management node, and size_from_bs is set to the file size returned by the distributed lock control node.

[0096] It should be understood that since file sizes do not change constantly, to reduce the communication overhead caused by worker nodes frequently retrieving file sizes from the resource management node when `trust_dn_size` is false, an expiration time for the file size retrieved from the resource management node can be set. This allows worker nodes to retrieve the file size from the resource management node again after the expiration time has passed, thereby reducing the number of communications between worker nodes and the resource management node and reducing communication overhead. The expiration time, i.e., the time interval, can be set according to actual conditions, and this embodiment does not impose any restrictions on it. For example, the time interval can be set to 10s, 30s, or 60s, etc.

[0097] That is, according to one embodiment of this disclosure, sending a read request to the resource management node may include:

[0098] A read request is sent to the resource management node when the time interval between the last time a read request was sent to the resource management node and the current time is greater than a preset time interval.

[0099] For example, the cache structure in the first index data table can be set to {valid_size, size_from_bs}, where valid_size is used to cache the file size sent by the distributed management node and the file size modified according to the modification request, and size_from_bs is used to cache the file size sent by the resource management node. The initial values ​​of both valid_size and size_from_bs are -1. When a worker node requests a write lock, the distributed lock control node returns the write lock along with the node validity flag trust_dn_size. If trust_dn_size is true, the worker node sets valid_size to the file size sent by the distributed lock control node; if trust_dn_size is false, the worker node does not set valid_size, i.e., the valid_size value remains -1. At this point, it is determined whether the time interval between the last time a read request was sent to the resource management node and the current time is greater than a preset time interval, such as greater than 30 seconds. If it is less than 30 seconds, the file size is not retrieved from the resource management node; if it is greater than 30 seconds, the file size is retrieved from the resource management node.

[0100] It should be understood that during the write lock holding period, a worker node may need to read the cached file size itself, and other worker nodes may also need to read the cached file size. Since the cached file size in a worker node can come from either the distributed lock control node or the resource management node, upon receiving a file size read request, it can first determine if the file size from the distributed lock control node is valid. If valid, the file size from the distributed lock control node is sent back to the worker node that needs to read the file size; if invalid, it then determines if the file size from the resource management node is valid. If valid, the file size from the resource management node is sent back to the worker node that needs to read the file size; otherwise, the file size is obtained from the resource management node.

[0101] That is, according to one embodiment of this disclosure, the method may further include:

[0102] After receiving the first write lock, in response to a request to read the file size of the target file, it determines whether the cached second file size is valid; if the second file size is valid, it outputs the second file size; if the second file size is invalid, it determines whether the cached third file size is valid; if the third file size is valid, it outputs the third file size; if the third file size is invalid, it obtains the file size of the target file from the resource management node.

[0103] Whether a cached file size is valid can be determined based on its specific value. For example, a negative file size indicates an invalid file size, while a positive file size indicates an invalid file size.

[0104] For example, if a file size of -1 indicates an invalid file size, then when a worker node needs to obtain the file size, such as when it receives a GetAttr request, it first checks `valid_size`. If `valid_size` is not -1, it is used; if `valid_size` is -1, it means the cached `valid_size` is invalid, and then `size_from_bs` is checked. If `size_from_bs` is not -1 and is within its validity period, it is used; if `size_from_bs` is -1 or has expired, it means `size_from_bs` is invalid. In this case, the file size needs to be obtained from the resource management node, and the file size sent by the resource management node is cached in `size_from_bs`.

[0105] As mentioned earlier, size_from_bs is used to cache the file size sent from the resource management node. Therefore, after modifying the file size cached in valid_size or size_from_bs, the file size cached in valid_size can be updated to the modified file size. This allows the file size cached in valid_size to be directly sent to the distributed lock management node when the write lock is revoked.

[0106] It's worth noting that in the abnormal scenario of write lock loss, only the cached file size in `size_from_bs` can be updated to the modified file size, not the cached file size in `valid_size`. This is because it's very possible that after the current worker node completes the `valid_size` update, the previous request to retrieve the file size hasn't yet reached the resource management node. In this case, the `valid_size` on the worker node's side would be incorrect. If the distributed lock management node withdraws the write lock after 60 seconds (the time after which the distributed lock management node cannot trust the received file size), then the distributed lock management node will accept this incorrect `valid_size`, leading to inconsistent file sizes. Therefore, in the scenario of write lock loss, the worker node can only update `size_from_bs`. When withdrawing the write lock, because `valid_size` is -1, it will retrieve the accurate file size from the resource management node and send it to the distributed lock management node.

[0107] It should be understood that during the period when a worker node holds a write lock, the updated file size is not immediately transmitted to the distributed lock management node. This means that the file size in the second index data table cannot accurately reflect the file size. In this situation, if other worker nodes want to read the file size, they must wait for the distributed lock management node to revoke the write lock and update the corresponding file size in the second index data table. Since the prerequisite for revoking the write lock is that the worker node holding the write lock does not need to use the write lock to change the file size at the current moment, otherwise, the first write lock cannot be revoked. This affects the efficiency of other worker nodes reading the file size. To overcome the above technical problem, this implementation records the target node information receiving the write lock, file size, and index node identifier when the distributed lock management node sends them to the worker nodes. This allows other worker nodes to send the target node information to the worker node that needs to read the file size, enabling that worker node to obtain the file size from the target worker node corresponding to the target node information. This eliminates the need to wait for the distributed lock management node to revoke the write lock before obtaining the file size, improving the efficiency of file size reading.

[0108] That is, according to one embodiment of this disclosure, the method can also be as follows: Figure 4 As shown, it may include:

[0109] In response to a read request for the file size of the target file, a second acquire request for acquiring a first read lock is sent to the distributed lock management node; if other worker nodes hold the first write lock, the system receives target node information representing the holding of the first write lock sent by the distributed lock management node in response to the second acquire request; the read request is sent to the target worker node corresponding to the target node information; when the target worker node responds to the read request, the system receives a fourth file size of the target file sent by the target worker node, the fourth file size being cached in the target worker node; when the target worker node does not respond to the read request, the system sends the read request to the resource management node; the system receives the third file size and caches the third file size in the target location of the first index data table according to the index node identifier.

[0110] In this case, the target worker node not responding to the read request could be due to a failure, restart, or the write lock on the target worker node being revoked the instant the read request was sent. If node information is obtained from the distributed lock management node in this situation, at least two communication connections must be established. Therefore, the file size of the target file can be directly obtained from the resource management node based on the first index node identifier, reducing the number of communications and lowering communication overhead.

[0111] The reason for not caching the file size obtained from the target worker node is that, since the target worker node holds a write lock, it may change the file size at any time, rendering the obtained file size invalid. Therefore, after obtaining the file size from the target worker node, it is only used temporarily and not cached.

[0112] In possible implementations, the method may further include:

[0113] The process involves: obtaining the node identifier of the working node; when the node identifier is a first node identifier, sending a file size check request to the distributed lock management node, wherein the first working node corresponding to the first node identifier is the working node used to check whether the file size in the second index data table is accurate; receiving the second index node identifier, a second read lock, and a fifth file size of the file to be checked sent by the distributed lock management node in response to the file size check request, wherein the second index node identifier is used to identify the index node of the file to be checked, and the second read lock is used to control the first working node's read operation on the file to be checked; obtaining the sixth file size of the file to be checked from the resource management node based on the second read lock and the second index node identifier; determining a check result to characterize whether the fifth file size and the sixth file size are the same based on the fifth file size and the sixth file size; and sending the check result to the distributed lock management node so that the distributed lock management node changes the node validity flag of the second index node identifier in the second index data table according to the check result.

[0114] The node identifier is used to identify the working node, and can be set according to the actual situation. This embodiment of the disclosure does not impose any restrictions on this. For example, the letter 'a' can be used to represent working node 1, and the letter 'b' can be used to represent working node 2, etc.

[0115] The sending of file size check requests to the distributed lock control node can be triggered automatically or manually, and can be sent periodically or non-periodically. This disclosure does not impose any restrictions on this.

[0116] For example, after sending a file size check request to the distributed lock management node, the first worker node receives the second inode identifier, the second read lock, and the fifth file size of the file to be checked from the distributed lock management node. Then, the first worker node first retrieves the sixth file size of the file to be checked from the third index data table in the resource management node based on the second read lock and the second inode identifier. It then compares the fifth file size with the sixth file size and determines the check result based on the comparison. Finally, it sends the check result to the distributed lock management node, so that the distributed lock management node can change the node validity flag of the second inode identifier in the second index data table according to the check result.

[0117] It is worth noting that, since the inode corresponding to the second inode identifier is the inode that needs to be checked for file size, the number of second inode identifiers in a single file size check can be determined according to the actual situation, and this embodiment does not impose any restrictions on this. Similarly, the number of second read locks, the number of fifth file sizes, and the number of check results correspond one-to-one with the number of second inode identifiers. For example, if the second inode identifier includes inode identifier 1, inode identifier 2, and inode identifier 3, then the second read locks can include read lock 1, read lock 2, and read lock 3, the fifth file size can include file size 1, file size 2, and file size 3, and the check results can include check result 1, check result 2, and check result 3. Furthermore, the second inode identifier, the second read lock, the fifth file size, and the check results correspond one-to-one, that is, inode identifier 1 corresponds to read lock 1, file size 1, and check result 1; inode identifier 2 corresponds to read lock 2, file size 2, and check result 2; and inode identifier 3 corresponds to read lock 3, file size 3, and check result 3.

[0118] Through the above technical solution, worker nodes can obtain the inode identifiers and file sizes that need to be checked from the distributed lock management node. They then retrieve the file sizes corresponding to these inode identifiers from the resource management node, compare them with the file sizes sent by the distributed lock management node to obtain the check results, and return the check results to the distributed lock management node. Thus, the distributed lock management node can proactively detect whether the file sizes in the second index data table are incorrect, thereby avoiding subsequent problems caused by errors in the file sizes in the second index data table.

[0119] In possible implementations, the method may further include:

[0120] Obtain the node identifier of the working node. When the node identifier is the second node identifier, send a file size correction request to the distributed lock management node. The second working node corresponding to the second node identifier is the working node used to correct the file size in the second index data table. Receive the second write lock and the third index node identifier sent by the distributed lock management node in response to the file size correction request. The second write lock is used to control the write operation of the second working node on the file to be corrected, and the third index node identifier is used to identify the index node of the file to be corrected. In response to the revocation request sent by the distributed lock management node to reclaim the second write lock, obtain the seventh file size of the file to be corrected from the third index data table in the resource management node according to the third index node identifier, and send the seventh file size to the distributed lock management node so that the distributed lock management node updates the file size of the file to be corrected in the second index data table to the seventh file size.

[0121] The sending of file size correction requests to the distributed lock control node can be triggered automatically or manually, and can be sent periodically or non-periodically. This disclosure does not impose any restrictions on this.

[0122] For example, after the second worker node sends a file size correction request to the distributed lock management node at time T, it receives the second write lock and the third index node identifier from the distributed lock management node at time T+1. Since the file size of the file to be corrected is incorrect, the distributed lock management node will not send the file size of the file to be corrected from the second index data table to the second worker node. Simultaneously, since the second write lock is actively issued by the distributed lock management node, the second worker node does not need to retrieve the corresponding file size from the resource management node based on the third index node identifier at time T+1. When the worker node receives the request to reclaim the second write lock at time T+2, since the valid_size corresponding to the third index node identifier in the first index data table is the default value (-1), the second worker node needs to retrieve the corresponding file size from the third index data table based on the third index node identifier and send it to the distributed lock management node so that the distributed lock management node updates the file size of the file to be corrected in the second index data table to the seventh file size.

[0123] It is worth noting that, since the inode corresponding to the third inode identifier is the inode that needs file size correction, the number of third inode identifiers in a single file size correction process can be determined according to the actual situation, and this embodiment does not impose any restrictions on this. Similarly, the number of second write locks corresponds one-to-one with the number of third inode identifiers. For example, if the third inode identifiers include inode identifier 1, inode identifier 2, and inode identifier 3, then the second write locks can include write lock 1, write lock 2, and write lock 3. Furthermore, the third inode identifiers and the second write locks correspond one-to-one, that is, inode identifier 1 corresponds to write lock 1, inode identifier 2 corresponds to write lock 2, and inode identifier 3 corresponds to write lock 3.

[0124] Since worker nodes incur significant performance overhead when retrieving file sizes from resource management nodes after write lock loss, this embodiment employs a file size repair mechanism. Specifically, worker nodes can obtain inode identifiers with `trust_dn_size` set to `false` from the distributed lock management node and record the write locks associated with these inode identifiers. When the distributed lock management node relinquishes these write locks, the corresponding file size is retrieved from the resource management node and sent to the distributed lock management node, enabling the distributed lock management node to repair the file size. Consequently, worker nodes can directly retrieve the file size corresponding to these inode identifiers from the distributed lock management node in subsequent accesses, reducing the performance loss caused by write lock loss.

[0125] Based on the same concept, this disclosure also provides a distributed data storage control method, which can be as follows: Figure 5 As shown, it may include:

[0126] S501: In response to a first acquisition request sent by a worker node for acquiring a first write lock, a first inode identifier, and a first file size, send the first write lock, the first file size, and the first inode identifier to the worker node. The first write lock is used to control the worker node's write operation on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file.

[0127] S502: In response to the first acquisition request sent by the other working nodes, send a revocation request to the working node to reclaim the first write lock.

[0128] S503: Receive the first write lock, the first index node identifier, and the second file size sent by the worker node in response to the revocation request; obtain the first file size in the second index data table according to the first index node identifier; update the first file size in the second index data table to the second file size; the second index data table is stored on the current node; the second file size is obtained by the worker node modifying the first file size sent by the distributed lock control node according to the first modification information in the first modification request; the first modification request is sent by the client.

[0129] The above technical solution allows for the following: upon reclaiming the write lock, the system receives the inode identifier and file size from the corresponding worker node and updates the file size corresponding to that inode identifier in the second index data table to match the file size sent by the worker node. Since the file size sent by the worker node is not immediately after modification but only upon receiving the write lock reclamation request, it avoids excessive requests to synchronize file size to the distributed lock management node in high-concurrency file size modification scenarios, thus preventing the distributed lock management node from experiencing performance and concurrency issues.

[0130] In possible implementations, the method may further include:

[0131] The system receives the first index node identifier, the second file size, and the first write lock identifier periodically sent by the working node; retrieves the second write lock identifier from the second index data table based on the first index node identifier, the second write lock identifier being used to identify the second write lock currently controlling the write operation on the target file; determines whether the first write lock identifier and the second write lock identifier are the same; and updates the first file size in the second index data table to the second file size when the first write lock identifier and the second write lock identifier are the same.

[0132] It should be understood that write locks and write lock identifiers generally have a one-to-one correspondence. By determining whether the second write lock identifier is the same as the first write lock identifier when updating the file size, update errors can be avoided.

[0133] For example, suppose worker node 1 sends a second file with a size of 10 during a certain sending cycle. Before the distributed lock management node receives the second file size sent by worker node 1, worker node 1 updates the second file size to 20. Simultaneously, the distributed lock management node reclaims write lock 1 from worker node 1. Since the second file size in worker node 1 is 20 when write lock 1 is reclaimed, the distributed lock management node updates the first file size in the second index data table to 20. At the same time, the distributed lock management node also allocates write lock 2, which controls worker nodes' write operations on the target file, to worker node 2. If the distributed lock management node only receives the second file size previously sent by worker node 1 at this point, without sending a write lock identifier, the distributed lock management node will change the first file size in the second index data table from 20 to 10, resulting in an update error.

[0134] To facilitate understanding of the periodic file size updates in this embodiment, the following provides a detailed explanation of the file size updates within an update cycle.

[0135] Suppose a worker node receives six inode identifiers from the distributed management system. Before the update time, the file sizes corresponding to inode identifiers 1 and 4 have changed. Therefore, a "dirty" flag is added to the file size 1 corresponding to inode identifier 1 in the first index data table, and a "dirty" flag is added to the file size 4 corresponding to inode identifier 4 in the first index data table. Inode identifier 1, write lock identifier 1, file size 1, write lock validity information 1, inode identifier 4, write lock identifier 4, file size 4, and write lock validity information 4 are then placed into the update queue. When the update time arrives, the worker node, based on the order of the inode identifiers in the update queue (e.g., inode identifier 1 is first), first checks the write lock validity information 1 to determine if write lock 1 is valid. If write lock 1 is valid, it sends inode identifier 1, write lock identifier 1, and file size 1 to the distributed lock management node. Upon receiving inode identifier 1, write lock identifier 1, and file size 1, the distributed lock management node first retrieves write lock identifier 3 from the second index data table based on inode identifier 1 and determines whether write lock identifier 3 is the same as write lock identifier 1. When write lock identifier 3 is the same as write lock identifier 1, the dirty flag is cleared, and the file size corresponding to inode identifier 1 in the second index data table is updated to the received file size 1. Then, the same processing steps are applied to inode identifier 4, write lock identifier 4, file size 4, and write lock validity information 4. If write lock 1 is invalid, inode identifier 1, write lock identifier 1, file size 1, and write lock validity information 1 are removed from the update queue, and the same processing steps are applied to inode identifier 4, write lock identifier 4, file size 4, and write lock validity information 4. Figure 6 As shown.

[0136] In possible implementations, the method may further include:

[0137] In response to a file size check request sent by a first working node, write lock information and node validity flag of an inode identifier are obtained. A second inode identifier is determined based on the write lock information and the node validity flag, and a fifth file size of the file to be checked is determined based on the second inode identifier. The write lock information indicates whether a write lock has been allocated to the file corresponding to the inode identifier, and the node validity flag indicates whether the file size corresponding to the inode identifier is valid. The file size check request is sent by the first working node, which is used to check whether the file size in the second index data table is accurate, and the node identifier of the first working node is a first node identifier. The second inode identifier, a second read lock, and the fifth file size are sent to the first working node. The second read lock controls the first working node's read operations on the file to be checked. A check result sent by the first working node indicating whether the fifth and sixth file sizes are the same is received, and the node validity flag of the second inode identifier is changed based on the check result. The sixth file size is obtained by the first working node from the resource management node based on the second read lock and the second inode identifier.

[0138] For example, at time T, the distributed lock management node responds to the file size check request sent by the first worker node. It first attempts to allocate read locks to each index node in the second index data table. If a write lock conflicting with the read lock is found, the index node is skipped. Simultaneously, to reduce unnecessary file size checks, the distributed lock management node can also check the node validity flag of the index nodes allocated read locks. If `trust_dn_size` is false, it indicates that the file size in the index node is invalid, and no file size check is needed. Thus, the second index node identifier can be obtained through these two operations. After determining the second index node identifier, the distributed lock management node sends the second index node identifier, the corresponding fifth file size, and the second read lock to the first worker node at time T+1, and receives the check result at time T+2. If the check result indicates that the fifth and sixth file sizes are the same, no action is taken. If the check result indicates that the fifth and sixth file sizes are different, it indicates that the fifth file size corresponding to the second index node identifier in the second index data table is invalid, and the `trust_dn_size` corresponding to the second index node identifier can be changed from true to false.

[0139] After the first worker node completes the file size check, it is highly likely that it will not need these second read locks. Therefore, to reduce the memory consumption of the second read locks, the distributed lock control node can initiate a revocation request for the allocated second read locks after receiving the check results.

[0140] It is worth noting that, since the inode corresponding to the second inode identifier is the inode that needs to be checked for file size, the number of second inode identifiers in a single file size check can be determined according to the actual situation, and this embodiment does not impose any restrictions on this. Similarly, the number of second read locks, the number of fifth file sizes, and the number of check results correspond one-to-one with the number of second inode identifiers. For example, if the second inode identifier includes inode identifier 1, inode identifier 2, and inode identifier 3, then the second read locks can include read lock 1, read lock 2, and read lock 3, the fifth file size can include file size 1, file size 2, and file size 3, and the check results can include check result 1, check result 2, and check result 3. Furthermore, the second inode identifier, the second read lock, the fifth file size, and the check results correspond one-to-one, that is, inode identifier 1 corresponds to read lock 1, file size 1, and check result 1; inode identifier 2 corresponds to read lock 2, file size 2, and check result 2; and inode identifier 3 corresponds to read lock 3, file size 3, and check result 3.

[0141] Through the above technical solution, worker nodes can obtain the inode identifiers and file sizes that need to be checked from the distributed lock management node. They then retrieve the file sizes corresponding to these inode identifiers from the resource management node, compare them with the file sizes sent by the distributed lock management node to obtain the check results, and return the check results to the distributed lock management node. Thus, the distributed lock management node can proactively detect whether the file sizes in the second index data table are incorrect, thereby avoiding subsequent problems caused by errors in the file sizes in the second index data table.

[0142] In a possible implementation, obtaining write lock information and node validity flag, and determining the second index node identifier based on the write lock information and the node validity flag, may include:

[0143] Based on the inspection progress in the first checklist, a third index node identifier is determined, wherein the first checklist is used to store the inspection progress of file size. The third index node identifier is used as the first traversal object. According to the preset first traversal order, the index node identifiers in the second index data table are traversed. For each index node identifier traversed, write lock information and node validity flag are obtained. When the write lock information indicates that the index node corresponding to the index node identifier has not been allocated a write lock, and the node validity flag indicates that the file size in the index node is valid, the index node identifier is determined as the second index node identifier.

[0144] The structure of the first checklist can be set according to the actual situation, and this embodiment does not impose any restrictions on it.

[0145] For example, the structure of the first check table can be: {file system identifier (fsid) of the currently reconciled system, inode identifier (inodeid), epoch, total number of inodes that have been reconciled, check time}. Here, epoch represents the round of checks, file system identifier is used to identify file systems (e.g., A1 identifies file system 1, A2 identifies file system 2, etc.), and inode identifier is used to identify inodes (e.g., A1-1 identifies the first inode in file system 1, A1-2 identifies the second inode in file system 1, A2-1 identifies the first inode in file system 2, etc.).

[0146] When determining the third inode identifier based on the first checklist, if the first checklist is empty, it indicates that the file size check is being performed for the first time. In this case, the first inode identifier among all inode identifiers in the distributed lock control node should be used as the third inode identifier. If the first checklist is not empty, and the file system identifiers do not contain all file system identifiers, it means that the check of all inode identifiers in the distributed lock control node has not been completed in this round. In this case, the third inode identifier can be determined based on the inode identifiers recorded in the inode identifiers and their order. If the first checklist is not empty, and the file system identifiers contain all file system identifiers, it means that the check of all inodes in the distributed lock control node has been completed in this round. In this case, the distributed lock control node records the completion of the current epoch and enters the next epoch. That is, the first inode identifier among all inode identifiers in the distributed lock control node is used as the third inode identifier again, or the file size check is not performed.

[0147] For example, if there are file systems A1 and A2, where file system A1 corresponds to inode identifiers A1-1, A1-2, and A1-3, and file system A2 corresponds to inode identifiers A2-1, A2-2, A2-3, and A2-4, and the order of all inode identifiers is A1-1, A1-2, A1-3, A2-1, A2-2, A2-3, and A2-4.

[0148] If the first check table is empty, then A1-1 can be used as the third inode identifier. If the first check table is represented as {A1, A1-1, A1-2, A1-3, A2-1, 0, 4, 13:20}, since the file system identifier does not contain A2, it means that there is an unchecked inode identifier in file system A2. At the same time, since the inode identifier records the inode identifiers as A1-1, A1-2, A1-3, and A2-1, it means that only A2-1 was checked last time. Therefore, this time the check can start from A2-2, that is, A2-2 can be used as the third inode identifier. If the first check table is represented as {A1, A2, A1-1, A1-2, A1-3, A2-1, A2-2, A2-3, A2-4, 0, 7, 13:25}, since the file system identifiers include A1 and A2, it means that the check of all inode identifiers in the distributed lock control node has been completed in this round. Therefore, the round number can be changed from 0 to 1, and A1-1 can be used as the third inode identifier again or no check can be performed.

[0149] To facilitate understanding of the file size check in this embodiment, the following is in conjunction with the appendix. Figure 7 The document provides a detailed explanation of the file size checking process.

[0150] At time T, the first worker node sends a file size check request to the distributed lock control node.

[0151] At time T+1, the distributed lock management node responds to the file size check request and then checks the first check table. If the first check table is empty, it starts traversing from the first inode identifier and, for each inode identifier encountered, determines whether its corresponding inode has a write lock allocated or whether trust_dn_size is false. If the inode identifier corresponding to this inode identifier has not been allocated a write lock and trust_dn_size is not false, then this inode identifier is determined as the second inode identifier, and a second read lock is allocated to its corresponding inode. Simultaneously, the inode identifier, the corresponding fifth file size, and the second read lock are recorded. If the first check table is not empty, it checks whether the current check round has completed the check of all inode identifiers in the distributed lock management node. If the current check round has completed the check of all inode identifiers in the distributed lock management node, no further action is taken. If the current check round has not completed the check of all index node identifiers in the distributed lock control node, the traversal starts from the position where the last check ended. For each index node identifier encountered, it is determined whether the index node corresponding to that identifier has a write lock allocated, or whether `trust_dn_size` is false. If the index node corresponding to that identifier has not been allocated a write lock and `trust_dn_size` is not false, then that index node identifier is determined as the second index node identifier, and a second read lock is allocated to its corresponding index node. Simultaneously, the index node identifier, the fifth file size corresponding to that index node identifier, and the second read lock are recorded. Furthermore, to avoid other working nodes modifying the file size corresponding to the second index node identifier in the second index data table during the file size check process, leading to inaccurate check results, the file size check time should not be too long. That is, the number of second index node identifiers, fifth file sizes, and second read locks in a single file size check should not be excessive. Therefore, during the recording of second index node identifiers, fifth file sizes, and second read locks, it is also possible to detect in real time whether the number of recorded second index node identifiers, fifth file sizes, or second read locks exceeds a threshold.

[0152] At time T+3, if the distributed lock control node detects that the second index node identifier, the fifth file size, or the number of second read locks is greater than the threshold, it will send all recorded second index node identifiers, fifth file sizes, and second read locks to the first worker node.

[0153] At time T+4, the first working node receives the second index node identifier, the fifth file size, and the second read lock from the distributed lock control node.

[0154] At time T+5, the first working node obtains the size of the sixth file corresponding to the second index node identifier from the third index data table based on the second read lock and the second index node identifier.

[0155] At time T+6, the first working node obtains the inspection result by comparing the size of the fifth file and the size of the sixth file.

[0156] At time T+7, the first working node sends the check result to the distributed lock control node.

[0157] At time T+8, the distributed lock control node receives the check result. If the check result indicates that the sizes of the fifth and sixth files are the same, no action is taken. If the check result indicates that the sizes of the fifth and sixth files are different, then `trust_dn_size` is changed from `true` to `false`.

[0158] In a possible implementation, to facilitate the management of inodes with trust_dn_size set to false, relevant information about inodes with trust_dn_size set to false can also be recorded in a second check table.

[0159] That is, according to one embodiment of this disclosure, the method may further include:

[0160] When the inspection results indicate that the size of the fifth file and the size of the sixth file are different, the file information corresponding to the second index node, the size of the fifth file, and the size of the sixth file are stored in the second inspection table.

[0161] The information related to inodes with `trust_dn_size` set to false can be determined based on actual circumstances, and this embodiment of the disclosure does not impose any restrictions on this. For example, the relevant information may include file information corresponding to the inode, the file size within the inode, and the inode identifier, etc.

[0162] The structure of the second check table can be set according to actual conditions, and this embodiment does not impose any restrictions on it. For example, the structure of the second check table can be: {fsid, inodeid, epoch, fifth file size, sixth file size}. Here, fsid is the file system identifier, used to identify the file system. For example, A1 can be used to identify file system 1, A2 can be used to identify file system 2, etc., inodeid is the inode identifier, and epoch represents the number of rounds of checks.

[0163] In possible implementations, the method may further include:

[0164] In response to a file size correction request sent by a second working node, a third index node identifier for the file to be corrected is determined, wherein the file size correction request is sent by the second working node, which is a working node used to correct the file size in the second index data table, and the node identifier of the second working node is the second node identifier; a second write lock and the third index node identifier are sent to the second working node, wherein the second write lock is used to control the writing operation of the working node on the file to be corrected; in response to a request to modify the file size of the file to be corrected, a revocation request for revoking the second write lock is sent to the second working node; a seventh file size, the third index node identifier, and the second write lock are received, and the file size of the file to be corrected in the second index data table is updated to the seventh file size, wherein the seventh file size is the seventh file size of the file to be corrected obtained by the second working node from the third index data table in the resource management node according to the third index node identifier in response to the revocation request.

[0165] For example, at time T, the distributed lock control node responds to the file size correction request by checking the node identifier corresponding to each inode identifier. If `trust_dn_size` is false, it means the file size corresponding to that inode identifier is invalid, and it is then identified as the third inode identifier. Afterwards, the distributed lock control node sends the second write lock and the third inode identifier to the second worker node. Since the file size corresponding to the third inode identifier is invalid, the distributed lock control node will not send the file size corresponding to the third inode identifier to the second worker node. When a modification request for the file size corresponding to the third inode identifier is received from other worker nodes at time T+1, a request to reclaim the second write lock is sent to the second worker node. If the seventh file size, the third inode identifier, and the second write lock are received from the second worker node at time T+3, the file size corresponding to the third inode identifier in the second index data table is updated to the seventh file size, and then the corresponding write lock and file size are sent to the worker nodes that need to modify the file size corresponding to the third inode identifier.

[0166] It is worth noting that, since the file size corresponding to the third inode identifier is invalid or incorrect, the number of third inode identifiers during a single file size correction process can be determined according to the actual situation, and this embodiment does not impose any restrictions on this. Similarly, the number of second write locks corresponds one-to-one with the number of third inode identifiers. For example, if the third inode identifiers include inode identifier 1, inode identifier 2, and inode identifier 3, then the second write locks can include write lock 1, write lock 2, and write lock 3. Furthermore, the third inode identifiers and the second write locks correspond one-to-one, that is, inode identifier 1 corresponds to write lock 1, inode identifier 2 corresponds to write lock 2, and inode identifier 3 corresponds to write lock 3.

[0167] Since worker nodes incur significant performance overhead when retrieving file sizes from resource management nodes after write lock loss, this embodiment employs a file size repair mechanism. Specifically, worker nodes can obtain inode identifiers with `trust_dn_size` set to `false` from the distributed lock management node and record the write locks associated with these inode identifiers. When the distributed lock management node relinquishes these write locks, the corresponding file size is retrieved from the resource management node and sent to the distributed lock management node, enabling the distributed lock management node to repair the file size. Consequently, worker nodes can directly retrieve the file size corresponding to these inode identifiers from the distributed lock management node in subsequent accesses, reducing the performance loss caused by write lock loss.

[0168] In a possible implementation, the file size correction request includes target information for determining the starting point of this correction, and the process of determining the third inode identifier of the file to be corrected in response to the file size correction request sent by the second working node may include:

[0169] The fourth index node identifier is determined based on the target information; the fourth index node identifier is used as the first traversal object, and the index node identifiers in the second index data table are traversed according to the preset second traversal order. For each index node identifier traversed, the node validity flag of the index node identifier is obtained, wherein the node validity flag is used to characterize whether the file size corresponding to the index node identifier is valid; when the node validity flag indicates that the file size corresponding to the index node identifier is invalid, the index node identifier is determined as the third index node identifier.

[0170] The target information can be an index node identifier or other information, and this embodiment of the disclosure does not impose any restrictions on it. The target information can be transmitted by the distributed lock management node to the second working node based on the previous correction information. This allows the distributed lock management node to avoid needing to check the validity of all file sizes in the second index data table when the second working node initiates a file size correction request again, thus improving correction efficiency.

[0171] The second traversal order can be the same as or different from the first traversal order, and this embodiment does not impose any restrictions on this.

[0172] To facilitate understanding of the file size correction in this embodiment, the following is in conjunction with the appendix. Figure 8 The process of correcting file size is explained in detail.

[0173] At time T, the second worker node sends a file size correction request to the distributed lock control node.

[0174] At time T+1, the distributed lock management node responds to the file size correction request. Then, based on the target information, it determines the fourth index node identifier and uses it as the first traversal object. Following a preset second traversal order, it traverses the index node identifiers in the second index data table. For each traversed index node identifier, it determines whether the corresponding index node has a write lock allocated, or whether `trust_dn_size` is false. If the index node has no write lock allocated and `trust_dn_size` is false, then the index node identifier is determined as the third index node identifier, and a second write lock is allocated to its corresponding index node. Simultaneously, the second write lock and the third index node identifier are recorded. Furthermore, to avoid the distributed lock management node consuming excessive communication resources when sending the second write lock and third index node identifier, thus affecting other communication processes, the number of second write locks and third index node identifiers sent in a single transaction should not be too large. Therefore, during the recording of the second write lock, the number of recorded second write locks or third index node identifiers can be monitored in real time to see if it exceeds a threshold.

[0175] At time T+3, if the distributed lock control node detects that the number of second write locks or third index node identifiers is greater than the threshold, it will send all the recorded second write locks, third index node identifiers, and the identifier of the index node that should be traversed first next time to the second worker node.

[0176] At time T+4, the second worker node receives the second write lock, the third index node identifier, and the identifier of the index node that should be traversed first next from the distributed lock control node.

[0177] At time T+5, the distributed lock control node responds to the modification request from the other working nodes regarding the file size corresponding to the third index node identifier, and sends a request to the second working node to reclaim the second write lock.

[0178] At time T+6, the second working node responds to the request to reclaim the second write lock and retrieves the corresponding seventh file size from the third index data table based on the second write lock and the third index node identifier.

[0179] At time T+7, the second working node sends the seventh file size, the third index node identifier, and the second write lock to the distributed lock management node.

[0180] At time T+8, the distributed lock control node receives the seventh file size, the second write lock, and the third index node identifier. It updates the file size corresponding to the third index node identifier in the second index data table to the seventh file size, changes the trust_dn_size corresponding to the third index node identifier from false to true, and sends the corresponding write lock and file size to the worker nodes that need to modify the file size corresponding to the third index node identifier.

[0181] Based on the same concept, embodiments of this disclosure also provide a distributed data storage control device, such as... Figure 9 As shown, the device may include:

[0182] The first sending module 901 is used to respond to the first modification request for the target file sent by the client, and send a first acquisition request to the distributed lock control node to acquire a first write lock, a first inode identifier, and a first file size. The first write lock is used to control the write operation of the worker node on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file.

[0183] The first modification module 902 is used to receive the first write lock, the first file size, and the first index node identifier sent by the distributed lock control node in response to the first acquisition request, modify the first file size according to the first modification information in the first modification request to obtain a second file size, and cache the second file size in the target position of the first index data table according to the first index node identifier. The first index data table is stored in the current working node.

[0184] The second sending module 903 is used to respond to the revocation request sent by the distributed lock management node for revoking the first write lock, send the first write lock, the first index node identifier, and the second file size to the distributed lock management node, so that the distributed lock management node updates the first file size corresponding to the first index node identifier in the second index data table to the second file size, and the second index data table is stored in the distributed lock management node.

[0185] In a possible implementation, the device may further include:

[0186] The second modification module, in response to the second modification request for the target file sent by the client, modifies the size of the second file according to the second modification information in the second modification request, obtains the modified size of the second file, and updates the size of the second file in the first index data table to the modified size of the second file.

[0187] In a possible implementation, the device may further include:

[0188] The first acquisition module is used to periodically acquire write lock validity information that characterizes whether the first write lock is valid;

[0189] The third sending module is used to send the first inode identifier, the second file size, and the first write lock identifier to the distributed lock management node when the write lock validity information indicates that the first write lock is valid. This allows the distributed lock management node to obtain the first file size and the second write lock identifier from the second index data table based on the first inode identifier, and update the first file size to the second file size based on the first write lock identifier and the second write lock identifier. The first write lock identifier is used to identify the first write lock, and the first write lock identifier is sent by the distributed lock management node when sending the first write lock to the worker node. The second write lock identifier is used to identify the second write lock currently controlling the write operation on the target file.

[0190] In a possible implementation, the device may further include:

[0191] The first receiving module is configured to receive, while receiving the first write lock, the first file size, and the first index node identifier, a node validity flag sent by the distributed lock control node, which is used to characterize whether the first file size is valid.

[0192] The first caching module is configured to cache the first file size in the target location of the first index data table according to the first index node identifier when the node validity flag indicates that the first file size is valid.

[0193] The fourth sending module is used to send a read request to the resource management node to read the file size of the target file when the node validity flag indicates that the file size is invalid, so that the resource management node can obtain the third file size of the target file from the third index data table according to the first index node identifier in the read request, and the third index data table is stored in the resource management node;

[0194] The second receiving module is used to receive the third file size and cache the third file size in the target position of the first index data table according to the inode identifier.

[0195] In a possible implementation, the device may further include:

[0196] The first determining module is configured to, after receiving the first write lock, determine whether the cached second file size is valid in response to a request to read the file size of the target file;

[0197] The first output module is used to output the size of the second file when the second file size is valid;

[0198] The second determining module is used to determine whether the cached third file size is valid when the second file size is invalid;

[0199] The second output module is used to output the third file size when the third file size is valid;

[0200] The second acquisition module is used to acquire the file size of the target file from the resource management node when the third file size is invalid.

[0201] In a possible implementation, the device may further include:

[0202] The fifth sending module is used to send a second acquisition request to the distributed lock control node in response to a request to read the file size of the target file, for acquiring the first read lock;

[0203] The third receiving module is used to receive, when there are other working nodes holding the first write lock, the information sent by the distributed lock control node in response to the second acquisition request, which is used to identify the target node holding the first write lock.

[0204] The sixth sending module is used to send the read request to the target working node corresponding to the target node information;

[0205] The fourth receiving module is configured to receive, when the target working node responds to the read request, a fourth file size of the target file sent by the target working node, wherein the fourth file size is cached in the target working node;

[0206] The seventh sending module is used to send the read request to the resource management node when the target working node does not respond to the read request;

[0207] The second caching module is used to receive the third file size and cache the third file size in the target position of the first index data table according to the inode identifier.

[0208] In a possible implementation, the seventh sending module can be used to send a read request to the resource management node when the time interval between the last time a read request was sent to the resource management node and the current time is greater than a preset time interval.

[0209] In a possible implementation, the device may further include:

[0210] The eighth sending module is used to obtain the node identifier of the working node, and when the node identifier is the first node identifier, it sends a file size check request to the distributed lock control node. The first working node corresponding to the first node identifier is the working node used to check whether the file size in the second index data table is accurate.

[0211] The fifth receiving module is used to receive the second index node identifier, the second read lock, and the fifth file size of the file to be checked sent by the distributed lock control node in response to the file size check request. The second index node identifier is used to identify the index node of the file to be checked, and the second read lock is used to control the first working node's read operation on the file to be checked.

[0212] The third acquisition module is used to obtain the sixth file size of the file to be inspected from the resource management node based on the second read lock and the second inode identifier;

[0213] The third determining module is used to determine a check result based on the size of the fifth file and the size of the sixth file to characterize whether the sizes of the fifth file and the sixth file are the same;

[0214] The ninth sending module is used to send the inspection result to the distributed lock management node, so that the distributed lock management node can change the node validity flag of the second index node identifier in the second index data table according to the inspection result.

[0215] In a possible implementation, the device may further include:

[0216] The tenth sending module is used to obtain the node identifier of the working node, and when the node identifier is the second node identifier, it sends a file size correction request to the distributed lock control node. The second working node corresponding to the second node identifier is the working node used to correct the file size in the second index data table.

[0217] The sixth receiving module is used to receive the second write lock and the third index node identifier sent by the distributed lock control node in response to the file size correction request. The second write lock is used to control the write operation of the second working node on the file to be corrected, and the third index node identifier is used to identify the index node of the file to be corrected.

[0218] The eleventh sending module is used to respond to the revocation request sent by the distributed lock management node for revoking the second write lock, obtain the seventh file size of the file to be corrected from the third index data table in the resource management node according to the third index node identifier, and send the seventh file size, the third index node identifier, and the second write lock to the distributed lock management node, so that the distributed lock management node updates the file size corresponding to the third index node identifier in the second index data table to the seventh file size.

[0219] Based on the same concept, embodiments of this disclosure also provide a distributed data storage control device, which can, as Figure 10 As shown, it includes:

[0220] The twelfth sending module 1001 is used to respond to a first acquisition request sent by a worker node for acquiring a first write lock, a first inode identifier, and a first file size, and send the first write lock, the first file size, and the first inode identifier to the worker node. The first write lock is used to control the worker node's write operation on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file.

[0221] The thirteenth sending module 1002 is used to send a revocation request to the working node to reclaim the first write lock in response to the first acquisition request sent by the other working nodes.

[0222] The first update module 1003 is used to receive the first write lock, the first index node identifier, and the second file size sent by the worker node in response to the revocation request, obtain the first file size in the second index data table according to the first index node identifier, and update the first file size in the second index data table to the second file size. The second index data table is stored on the current node, and the second file size is obtained by the worker node modifying the first file size sent by the distributed lock control node according to the first modification information in the first modification request. The first modification request is sent by the client.

[0223] In a possible implementation, the device may further include:

[0224] The seventh receiving module is used to receive the first index node identifier, the second file size, and the first write lock identifier periodically sent by the working node;

[0225] The fourth acquisition module is used to acquire a second write lock identifier from the second index data table based on the first inode identifier. The second write lock identifier is used to identify the second write lock currently controlling the write operation on the target file.

[0226] The fourth determining module is used to determine whether the first write lock identifier and the second write lock identifier are the same;

[0227] The second update module is used to update the first file size in the second index data table to the second file size when the first write lock identifier and the second write lock identifier are the same.

[0228] In a possible implementation, the device may further include:

[0229] The fifth determining module is used to respond to a file size check request sent by the first working node, obtain write lock information and node validity flag of the inode identifier, determine the second inode identifier based on the write lock information and the node validity flag, and determine the fifth file size of the file to be checked based on the second inode identifier. The write lock information is used to indicate whether the file corresponding to the inode identifier has been allocated a write lock, and the node validity flag is used to indicate whether the file size corresponding to the inode identifier is valid. The file size check request is sent by the first working node, which is a working node used to check whether the file size in the second index data table is accurate, and the node identifier of the first working node is the first node identifier.

[0230] The fourteenth sending module is used to send the second index node identifier, the second read lock, and the fifth file size to the first working node. The second read lock is used to control the first working node's read operation on the file to be checked.

[0231] The modification module is used to receive the check result sent by the first working node, which indicates whether the size of the fifth file and the size of the sixth file are the same, and to modify the node validity flag of the second index node identifier according to the check result. The size of the sixth file is obtained by the first working node from the resource management node based on the second read lock and the second index node identifier.

[0232] In a possible implementation, the fifth determining module may include:

[0233] The first determining unit is used to determine the third index node identifier based on the inspection progress in the first check table, wherein the first check table is used to store the inspection progress of file size.

[0234] The second determining unit is used to take the third index node identifier as the first traversal object, traverse the index node identifiers in the second index data table according to the preset first traversal order, and for each traversed index node identifier, obtain write lock information and node validity flag. When the write lock information indicates that the index node corresponding to the index node identifier has not been allocated a write lock, and the node validity flag indicates that the file size in the index node is valid, the index node identifier is determined as the second index node identifier.

[0235] In a possible implementation, the device may further include:

[0236] The sixth determining module is used to determine the third index node identifier of the file to be corrected in response to a file size correction request sent by the second working node, wherein the file size correction request is sent by the second working node, the second working node is a working node used to correct the file size in the second index data table, and the node identifier of the second working node is the second node identifier.

[0237] The fifteenth sending module is used to send the second write lock and the third index node identifier to the second working node, wherein the second write lock is used to control the working node's write operation on the file to be corrected;

[0238] The sixteenth sending module is used to send a revocation request to the second working node in response to a request to modify the file size of the file to be corrected, for revoking the second write lock.

[0239] The third update module is used to receive the seventh file size, the third index node identifier, and the second write lock, and update the file size corresponding to the third index node identifier in the second index data table to the seventh file size, wherein the seventh file size is the seventh file size of the file to be corrected obtained by the second working node from the third index data table in the resource management node according to the third index node identifier in response to the revocation request.

[0240] In a possible implementation, the file size correction request includes target information for determining the starting point of this correction; correspondingly, the sixth determining module may include:

[0241] The third determining unit is used to determine the fourth index node identifier based on the target information;

[0242] The acquisition unit is used to take the fourth index node identifier as the first traversal object, traverse the index node identifiers in the second index data table according to the preset second traversal order, and for each index node identifier traversed, acquire the node validity flag of the index node identifier, wherein the node validity flag is used to characterize whether the file size corresponding to the index node identifier is valid.

[0243] The fourth determining unit is used to determine the index node identifier as the third index node identifier when the node validity flag indicates that the file size corresponding to the index node identifier is invalid.

[0244] Based on the same concept, embodiments of this disclosure also provide a computer-readable medium having a computer program stored thereon that, when executed by a processing device, implements the steps of the method described in any one of the first or second aspects.

[0245] Based on the same concept, this disclosure also provides an electronic device that may include:

[0246] A storage device on which computer programs are stored;

[0247] A processing device for executing the computer program in the storage device to implement the steps of the method in either the first or second aspect.

[0248] The following is for reference. Figure 11The diagram illustrates a structural schematic of an electronic device 1100 suitable for implementing embodiments of the present disclosure. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0249] like Figure 11 As shown, electronic device 1100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1102 or a program loaded from storage device 1108 into random access memory (RAM) 1103. The RAM 1103 also stores various programs and data required for the operation of electronic device 1100. The processing device 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0250] Typically, the following devices can be connected to I / O interface 1105: input devices 1106 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1107 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1108 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1109. Communication device 1109 allows electronic device 1100 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 11 An electronic device 1100 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0251] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1109, or installed from storage device 1108, or installed from ROM 1102. When the computer program is executed by processing device 1101, it performs the functions defined in the methods of embodiments of this disclosure.

[0252] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0253] In some implementations, communication can be conducted using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can be interconnected with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0254] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0255] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, enable the electronic device to perform the steps of the method described in either the first or second aspect.

[0256] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, enable the electronic device to perform the steps of the method described in either the first or second aspect.

[0257] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0258] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0259] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0260] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0261] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0262] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0263] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0264] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. A distributed data storage control method, characterized in that, The method includes: In response to the first modification request for the target file sent by the client, a first acquisition request is sent to the distributed lock control node to acquire the first write lock, the first inode identifier, and the first file size. The first write lock is used to control the write operation of the worker node on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file. The system receives the first write lock, the first file size, and the first index node identifier returned by the distributed lock control node. It modifies the first file size according to the first modification information in the first modification request to obtain the second file size. It then caches the second file size in the target position of the first index data table according to the first index node identifier. The first index data table is stored in the current working node. In response to the revocation request sent by the distributed lock management node for revoking the first write lock, the first write lock, the first index node identifier, and the second file size are sent to the distributed lock management node, so that the distributed lock management node updates the first file size corresponding to the first index node identifier in the second index data table to the second file size, and the second index data table is stored in the distributed lock management node; Receive a node validity flag sent by the distributed lock control node to indicate whether the size of the first file is valid; When the node validity flag indicates that the first file size is valid, the first file size is cached in the target location of the first index data table according to the first index node identifier; When the node validity flag indicates that the first file size is invalid, a read request for reading the file size of the target file is sent to the resource management node; The system receives the third file size of the target file from the third index data table returned by the resource management node, and caches the third file size in the target location of the first index data table according to the index node identifier. The third index data table is a data table stored in the resource management node for storing file index data.

2. The method according to claim 1, characterized in that, The method further includes: Periodically acquire write lock validity information to characterize whether the first write lock is valid; When the write lock validity information indicates that the first write lock is valid, the first index node identifier, the second file size, and the first write lock identifier are sent to the distributed lock management node so that the distributed lock management node updates to the second file size.

3. The method according to claim 1, characterized in that, The method further includes: After receiving the first write lock, in response to a request to read the file size of the target file, it is determined whether the cached second file size is valid; If the second file size is valid, output the second file size; If the second file size is invalid, determine whether the cached third file size is valid; If the third file size is valid, output the third file size; If the third file size is invalid, obtain the file size of the target file from the resource management node.

4. The method according to claim 3, characterized in that, The method further includes: In response to a request to read the file size of the target file, a second acquisition request for acquiring the first read lock is sent to the distributed lock control node; When there are other working nodes holding the first write lock, receive the target node information representing the holding of the first write lock returned by the distributed lock control node; Send the read request to the target worker node corresponding to the target node information; When the target worker node responds to the read request, the fourth file size of the target file sent by the target worker node is received, and the fourth file size is cached in the target worker node; If the target working node does not respond to the read request, the read request is sent to the resource management node; The third file size is received, and the third file size is cached in the target location of the first index data table according to the inode identifier.

5. The method according to claim 4, characterized in that, Sending a read request to the resource management node includes: A read request is sent to the resource management node when the time interval between the last time a read request was sent to the resource management node and the current time is greater than a preset time interval.

6. The method according to any one of claims 1-3, characterized in that, The method further includes: When the node identifier of the working node indicates that the working node is the first working node used to check whether the file size in the second index data table is accurate, a file size check request is sent to the distributed lock control node; The system receives a second index node identifier, a second read lock, and a fifth file size of the file to be checked, returned by the distributed lock control node. The second index node identifier is used to identify the index node of the file to be checked, and the second read lock is used to control the first working node's read operation on the file to be checked. The fifth file size is determined based on the second index node identifier. Based on the second read lock and the second index node identifier, the sixth file size of the file to be inspected is obtained from the third index data table in the resource management node; wherein, the sixth file size is obtained by the first working node from the resource management node based on the second read lock and the second index node identifier; Based on the size of the fifth file and the size of the sixth file, a check result is determined to characterize whether the sizes of the fifth file and the sixth file are the same; wherein, the second inode identifier, the second read lock, the size of the fifth file and the check result correspond one-to-one; The inspection result is sent to the distributed lock management node, so that the distributed lock management node changes the node validity flag of the second index node identifier in the second index data table according to the inspection result.

7. The method according to any one of claims 1-3, characterized in that, The method further includes: When the node identifier of the working node indicates that the working node is a second working node used to correct the file size in the second index data table, a file size correction request is sent to the distributed lock control node; The system receives a second write lock and a third index node identifier returned by the distributed lock control node. The second write lock is used to control the write operation of the second working node on the file to be corrected, and the third index node identifier is used to identify the index node of the file to be corrected. In response to a revocation request sent by the distributed lock management node to reclaim the second write lock, the seventh file size of the file to be corrected is obtained from the third index data table in the resource management node according to the third index node identifier. The seventh file size, the third index node identifier, and the second write lock are then sent to the distributed lock management node, so that the distributed lock management node updates the file size corresponding to the third index node identifier in the second index data table to the seventh file size. The seventh file size is obtained by the second working node from the third index data table in the resource management node according to the third index node identifier in response to the revocation request.

8. A distributed data storage control method, characterized in that, The method includes: In response to a first acquisition request sent by a worker node for acquiring a first write lock, a first inode identifier, and a first file size, the first write lock, the first file size, and the first inode identifier are sent to the worker node. The first write lock is used to control the worker node's write operations on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file. In response to the first acquisition request sent by the other working nodes, a revocation request for revoking the first write lock is sent to the working node; The system receives the first write lock, the first index node identifier, and the second file size returned by the worker node. Based on the first index node identifier, it obtains the first file size in the second index data table and updates the first file size in the second index data table to the second file size. The second index data table is stored in the current node. The second file size is obtained by the worker node modifying the first file size sent by the distributed lock control node according to the first modification information in the first modification request. The first modification request is sent by the client. In response to a file size check request sent by a first working node, write lock information and node validity flag of an inode identifier are obtained. A second inode identifier is determined based on the write lock information and the node validity flag. A fifth file size of the file to be checked is then determined based on the second inode identifier. The write lock information indicates whether a write lock has been allocated to the file corresponding to the inode identifier, and the node validity flag indicates whether the file size corresponding to the inode identifier is valid. The file size check request is sent by the first working node, which is used to check whether the file size in the second index data table is accurate. The fifth file size is determined based on the second inode identifier. Send the second inode identifier, the second read lock, and the fifth file size to the first working node. The second read lock is used to control the first working node's read operation on the file to be inspected. The system receives a check result from the first working node indicating whether the sizes of the fifth and sixth files are the same, and modifies the node validity flag of the second index node identifier based on the check result. The size of the sixth file is obtained by the first working node from the resource management node based on the second read lock and the second index node identifier. The second index node identifier, the second read lock, the size of the fifth file, and the check result correspond one-to-one.

9. The method according to claim 8, characterized in that, The method further includes: Receive the first index node identifier, the second file size, and the first write lock identifier periodically sent by the working node; The second write lock identifier is obtained from the second index data table based on the first inode identifier. The second write lock identifier is used to identify the second write lock currently controlling the write operation on the target file. Determine whether the first write lock identifier and the second write lock identifier are the same; When the first write lock identifier and the second write lock identifier are the same, the first file size in the second index data table is updated to the second file size.

10. The method according to claim 9, characterized in that, The step of obtaining the write lock information and node validity flag of the inode identifier, and determining the second inode identifier based on the write lock information and the node validity flag, includes: The third index node identifier is determined based on the check progress in the first check table, wherein the first check table is used to store the check progress of file size. Using the third index node identifier as the first traversal object, the index node identifiers in the second index data table are traversed according to the preset first traversal order. For each index node identifier traversed, write lock information and node validity flag are obtained. When the write lock information indicates that the index node corresponding to the index node identifier has not been allocated a write lock, and the node validity flag indicates that the file size in the index node is valid, the index node identifier is determined as the second index node identifier.

11. The method according to claim 8 or 9, characterized in that, The method further includes: In response to a file size correction request sent by a second working node, a third index node identifier for the file to be corrected is determined, wherein the file size correction request is sent by the second working node, and the second working node is a working node used to correct the file size in the second index data table; The second write lock and the third index node identifier are sent to the second working node, wherein the second write lock is used to control the working node's write operations on the file to be corrected; In response to the request to modify the file size of the file to be corrected, a revocation request for revoking the second write lock is sent to the second working node; The system receives the seventh file size, the third index node identifier, and the second write lock, and updates the file size corresponding to the third index node identifier in the second index data table to the seventh file size. The seventh file size is the seventh file size of the file to be corrected obtained by the second working node from the third index data table in the resource management node according to the third index node identifier in response to the revocation request.

12. The method according to claim 11, characterized in that, The file size correction request includes target information for determining the starting point of this correction. The process of determining the third inode identifier of the file to be corrected in response to the file size correction request sent by the second working node includes: The fourth index node identifier is determined based on the target information; The fourth index node identifier is used as the first traversal object. According to the preset second traversal order, the index node identifiers in the second index data table are traversed. For each index node identifier traversed, the node validity flag of the index node identifier is obtained. The node validity flag is used to characterize whether the file size corresponding to the index node identifier is valid. When the node validity flag indicates that the file size corresponding to the inode identifier is invalid, the inode identifier is determined as the third inode identifier.

13. A distributed data storage control device, characterized in that, The device includes: The first sending module is used to respond to the first modification request for the target file sent by the client, and send a first acquisition request to the distributed lock control node to acquire a first write lock, a first inode identifier, and a first file size. The first write lock is used to control the write operation of the worker node on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file. The first modification module is used to receive the first write lock, the first file size, and the first index node identifier returned by the distributed lock control node, modify the first file size according to the first modification information in the first modification request to obtain the second file size, and cache the second file size in the target position of the first index data table according to the first index node identifier. The first index data table is stored in the current working node. The second sending module is used to respond to the revocation request sent by the distributed lock management node for revoking the first write lock, send the first write lock, the first index node identifier, and the second file size to the distributed lock management node, so that the distributed lock management node updates the first file size corresponding to the first index node identifier in the second index data table to the second file size, and the second index data table is stored in the distributed lock management node; The first receiving module is used to receive a node validity flag sent by the distributed lock control node to characterize whether the first file size is valid. The first caching module is configured to cache the first file size in the target location of the first index data table according to the first index node identifier when the node validity flag indicates that the first file size is valid. The fourth sending module is used to send a read request to the resource management node to read the file size of the target file when the node validity flag indicates that the first file size is invalid; The second receiving module is used to receive the third file size of the target file in the third index data table returned by the resource management node, and cache the third file size in the target position of the first index data table according to the index node identifier. The third index data table is a data table stored in the resource management node for storing file index data, and the third file size is the file size obtained from the resource management node.

14. A distributed data storage control device, characterized in that, The device includes: The twelfth sending module is used to respond to a first acquisition request sent by a worker node for acquiring a first write lock, a first inode identifier, and a first file size, and send the first write lock, the first file size, and the first inode identifier to the worker node. The first write lock is used to control the worker node's write operation on the target file, the first inode identifier is used to identify the inode of the target file, and the first file size is the file size of the target file. The thirteenth sending module is used to send a revocation request to the working node to reclaim the first write lock in response to the first acquisition request sent by the other working nodes. The first update module is used to receive the first write lock, the first index node identifier, and the second file size returned by the worker node, obtain the first file size in the second index data table according to the first index node identifier, and update the first file size in the second index data table to the second file size. The second index data table is stored in the current node. The second file size is obtained by the worker node modifying the first file size sent by the distributed lock control node according to the first modification information in the first modification request. The first modification request is sent by the client. The fifth determining module is used to respond to a file size check request sent by the first working node, obtain write lock information and node validity flag of the inode identifier, determine a second inode identifier based on the write lock information and the node validity flag, and determine a fifth file size of the file to be checked based on the second inode identifier. The write lock information indicates whether a write lock has been allocated to the file corresponding to the inode identifier, and the node validity flag indicates whether the file size corresponding to the inode identifier is valid. The file size check request is sent by the first working node, which is used to check whether the file size in the second index data table is accurate. The fifth file size is determined based on the second inode identifier. The fourteenth sending module is used to send the second index node identifier, the second read lock, and the fifth file size to the first working node. The second read lock is used to control the first working node's read operation on the file to be checked. The modification module is used to receive the check result sent by the first working node, which indicates whether the size of the fifth file and the size of the sixth file are the same, and modify the node validity flag of the second index node identifier according to the check result. The size of the sixth file is obtained by the first working node from the resource management node according to the second read lock and the second index node identifier. The second index node identifier, the second read lock, the size of the fifth file and the check result correspond one-to-one.

15. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processing device, it implements the steps of the method described in any one of claims 1-12.

16. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-12.

Images