File lock processing method and device, electronic equipment and storage medium
By establishing a lock request and response mechanism between the file flow client and the metadata server, the problem of the client misjudging the file locking status is solved, ensuring the atomicity and consistency of the file write operation, and avoiding deadlocks.
Patent Information
- Application Number
- CN202411773088.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-04-11
AI Technical Summary
In the database, when the client locks the file before writing, the client may misjudgment whether the file is locked, resulting in inconsistent with the actual locking state of the database, which will cause deadlock.
By establishing a lock request and response mechanism between the file flow client and the metadata server, ensure that the client correctly acquires the lock status before the write operation. The specific steps include the client obtaining a write request, generating a lock request, sending a request to the metadata server, the metadata server detects the file status and returns a response, and the client stores response information to confirm the file lock status.
It effectively solves the problem of client misjudging file locking status, ensures the atomicity and consistency of file write operations, and avoids the occurrence of deadlocks.
Smart Images
Figure CN119807151A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of cloud computing technology, and in particular to a method, device, electronic device and storage medium for processing file locks. Background Art
[0002] When writing to a file in the database, in order to achieve file consistency and atomicity, only one client is supported to write to the file at a time. To this end, before a client writes to a file, the database needs to lock the file for the client, and then the client can write to the file. During the process of the client writing to the file, other clients cannot write to the file because the file is locked. After the client completes the write operation to the file, the database unlocks the file, and other clients can subsequently write to the file.
[0003] However, the inventors found that in the above situation, sometimes the client believes that the file is not locked but the database has actually locked the file, resulting in the client's conclusion that whether the file is locked is inconsistent with the objective fact that the database has actually locked the file. Summary of the invention
[0004] The present application shows a method, device, electronic device and storage medium for processing file locks.
[0005] In a first aspect, the present application shows a method for processing a file lock, which is applied to a file stream client, and the method includes:
[0006] Obtaining a first write request for a target file submitted by a user, where the first write request carries at least an ID of the target file;
[0007] Obtain a first lock sequence number according to the first write request;
[0008] Generate a first lock request for the target file, the first lock request carrying the ID of the file stream client, the first lock sequence number, and the ID of the target file;
[0009] Sending a first lock request to a metadata server; allowing the metadata server to receive the first lock request for a target file sent by a file stream client; detecting whether the target file has been locked according to the ID of the target file; storing in a persistent storage space: a correspondence between the ID of the file stream client, the first lock sequence number, and the ID of the target file if the target file has not been locked; locking the target file according to the ID of the target file; and returning a lock response to the file stream client, the lock response being used to indicate that the target file has been locked;
[0010] Receive the lock response returned by the metadata server;
[0011] Information indicating that the target file has been locked is stored according to the lock response.
[0012] In a second aspect, the present application shows a method for processing a file lock, which is applied to a metadata server, and the method includes:
[0013] Receive a first lock request for a target file sent by a file stream client, the first lock request carrying an ID of the file stream client, a first lock sequence number, and an ID of the target file; the first lock sequence number is obtained by the file stream client according to a first write request for the target file submitted by a user, the first write request carrying at least the ID of the target file; the first lock request is generated by the file stream client;
[0014] According to the ID of the target file, check whether the target file has been locked;
[0015] When the target file is not locked, the corresponding relationship between the ID of the file stream client, the first locking sequence number and the ID of the target file is stored in the persistent storage space;
[0016] Lock the target file according to its ID;
[0017] A locking response is returned to the file stream client, where the locking response is used to indicate that the target file has been locked, so that the file stream client stores information indicating that the target file has been locked according to the locking response.
[0018] In a third aspect, the present application shows a file lock processing device, which is applied to a file stream client, and the device includes:
[0019] A first acquisition module, used to acquire a first write request for a target file submitted by a user, wherein the first write request carries at least an ID of the target file;
[0020] A second acquisition module, used to acquire a first locking sequence number according to the first write request;
[0021] A first generating module, used to generate a first locking request for a target file, wherein the first locking request carries an ID of a file stream client, a first locking sequence number, and an ID of the target file;
[0022] The first sending module is used to send a first locking request to the metadata server; so that the metadata server receives the first locking request for the target file sent by the file stream client; detects whether the target file has been locked according to the ID of the target file; if the target file is not locked, stores in the persistent storage space: a correspondence between the ID of the file stream client, the first locking sequence number and the ID of the target file; locks the target file according to the ID of the target file; and returns a locking response to the file stream client, where the locking response is used to indicate that the target file has been locked;
[0023] A first receiving module, used to receive a lock response returned by the metadata server;
[0024] The first storage module is used to store information indicating that the target file has been locked according to the locking response.
[0025] In a fourth aspect, the present application shows a file lock processing device, which is applied to a metadata server, and the device includes:
[0026] A second receiving module is used to receive a first lock request for a target file sent by a file stream client, wherein the first lock request carries an ID of the file stream client, a first lock sequence number, and an ID of the target file; the first lock sequence number is obtained by the file stream client according to a first write request for the target file submitted by a user, wherein the first write request carries at least the ID of the target file; the first lock request is generated by the file stream client;
[0027] A detection module, used to detect whether the target file has been locked according to the ID of the target file;
[0028] The second storage module is used to store, in the persistent storage space, the corresponding relationship between the ID of the file stream client, the first locking sequence number and the ID of the target file when the target file is not locked;
[0029] A locking module is used to lock the target file according to the ID of the target file;
[0030] The return module is used to return a locking response to the file stream client, where the locking response is used to indicate that the target file has been locked, so that the file stream client stores information indicating that the target file has been locked according to the locking response.
[0031] In a fifth aspect, the present application illustrates an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the method described in any of the above aspects.
[0032] In a sixth aspect, the present application illustrates a non-temporary computer-readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute a method as described in any of the above aspects.
[0033] In a seventh aspect, the present application illustrates a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in any of the above aspects.
[0034] The technical solution provided by this application may have the following beneficial effects:
[0035] In the present application, between "the file stream client obtains the first write request for the target file submitted by the user" to "when the target file is not locked, the metadata server stores in the persistent storage space: the correspondence between the ID of the file stream client, the first locking sequence number and the ID of the target file", if the metadata server fails (or crashes), the metadata server will not lock the target file, and the file stream client will not receive the locking response sent by the metadata server to indicate that the target file has been locked, and further the file stream client will not store the information indicating that the target file has been locked.
[0036] In this way, since the file stream client did not receive the locking response returned by the metadata server, the file stream client will believe that the target file is not locked. Even if other normal metadata servers subsequently replace the downed metadata server, since there is no "correspondence between the file stream client's ID, the first locking sequence number and the target file's ID" in the persistent storage space, other normal metadata servers will believe that the target file is not locked. It can be seen that the file stream client and other normal metadata servers will believe that the target file is not locked, so that the file stream client and other normal metadata servers have the same understanding of whether the target file is locked, avoiding deadlock. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 It is a schematic diagram of a file lock processing system of the present application.
[0038] Figure 2 This is a schematic diagram of a directory fragmentation of the present application.
[0039] Figure 3 It is a flowchart of the steps of a method for processing file locks of the present application.
[0040] Figure 4 It is a structural block diagram of a file lock processing device of the present application.
[0041] Figure 5 It is a structural block diagram of a file lock processing device of the present application.
[0042] Figure 6 It is a block diagram of an electronic device of the present application.
[0043] Figure 7 It is a block diagram of an electronic device of the present application. DETAILED DESCRIPTION
[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0045] Reference Figure 1 , showing a structural block diagram of a file lock processing system of the present application.
[0046] The system is built on a distributed Store Cluster and provides application services with Append Only file operation semantics for upper-level applications used by users.
[0047] The system includes: file stream client (FS Client, File Stream Client), file stream controller (FS Master, File Stream Master), metadata server (MDS, Meta Data Server), file block client (Chunk Client) and storage cluster (Store Cluster).
[0048] The file stream client provides the upper-layer application with the file operation semantics of Append Only, including the operation interfaces of Create, Delete, Open / Close, Read, and Append. The file system obtains the file system view through the file stream controller, operates the file metadata through the metadata server, and persists the file data to the Store Cluster.
[0049] The file flow controller manages the life cycle of each node in the file system cluster by maintaining the heartbeat between clusters, scheduling logical file system directory shards, and maintaining the file system view.
[0050] FS Master dispatches the directory fragments of each logical file system to the corresponding metadata server according to a certain algorithm. For example, Figure 2 shown.
[0051] The metadata server loads and maintains the directory tree and file status of the specified file system shards and provides metadata read and write interfaces. The metadata server uses Store Cluster to persist the directory shards and ensures the atomicity and consistency of metadata modifications through WAL (Write Ahead Log). It also provides a file system operation interface to the FS Client.
[0052] The file block client serves as the upper-layer interface of the Store Cluster and provides basic data persistence capabilities.
[0053] The storage cluster uses Chunk (file block) as the basic operation unit and provides basic operations such as Read / Append / Seal for the specified Chunk ID (Identity Document). After the Seal operation is successfully executed, the Chunk will be permanently unreadable and unwritable.
[0054] Reference Figure 3 , shows a flowchart of the steps of a file lock processing method of the present application, wherein the method is applied to Figure 1 The file stream client and metadata server shown in FIG. 1 include:
[0055] In step S101, the file stream client obtains a first write request for a target file submitted by a user, where the first write request carries at least an ID of the target file.
[0056] The first write request to the target file submitted by the user may be via Figure 1 The application used by the user shown in the figure submits the file to the file stream client.
[0057] The target file is located at Figure 1 In the Store Cluster in the system shown.
[0058] In step S102, the file stream client obtains a first locking sequence number according to a first write request.
[0059] The lock sequence number can be sessionID, etc.
[0060] In this application, any two locking sequence numbers are different.
[0061] For example, a locking sequence number library may be generated in advance, wherein the locking sequence number library contains a plurality of locking sequence numbers, and any two locking sequence numbers in the locking sequence number library are different.
[0062] To this end, when the file stream client obtains the first locking sequence number according to the first write request, a locking sequence number can be extracted from the locking sequence number library and used as the first locking sequence number.
[0063] It should be noted that after a locking sequence number is extracted from the locking sequence number library, the locking sequence number is no longer in the locking sequence number library.
[0064] In step S103, the file stream client generates a first lock request for the target file, and the first lock request carries the ID of the file stream client, the first lock sequence number, and the ID of the target file.
[0065] The first lock request is used to lock the target file.
[0066] In step S104, the file stream client sends a first lock request to the metadata server.
[0067] Although the target file is located at Figure 1 In the Store Cluster in the system shown, the directory (storage path) of the target file in the StoreCluster is located in the metadata server. Therefore, in order to obtain the target file, the file stream client needs to first obtain the directory of the target file in the Store Cluster from the metadata server.
[0068] To this end, the file stream client needs to obtain the first lock sequence number based on the first write request, generate a first lock request for the target file, the first lock request carries the ID of the file stream client, the first lock sequence number and the ID of the target file, and sends the first lock request to the metadata server.
[0069] In step S105, the metadata server receives a first lock request for the target file sent by the file stream client.
[0070] In step S106, the metadata server detects whether the target file has been locked according to the ID of the target file.
[0071] In one embodiment of the present application, if the metadata server locks a file in the Store Cluster, the metadata server may store the ID of the file in the lock list to indicate that the file has been locked, or may record a specific character (used to indicate that the file has been locked) in a specific field in the directory of the file. The specific character may be a number "1" or "2", etc.
[0072] If the metadata server unlocks the file in the Store Cluster later, the metadata server can delete the file ID from the lock list to indicate that the file has been unlocked, or delete the specific characters recorded in a specific field in the directory of the file (the absence of specific characters in the specific field is used to indicate that the file is not locked).
[0073] Thus, in this step, when the metadata server detects whether the target file has been locked based on the ID of the target file, the ID of the target file can be searched in the lock list. If the ID of the target file is found in the lock list, it can be determined that the target file has been locked. Alternatively, if the ID of the target file is not found in the lock list, it can be determined that the target file is not locked.
[0074] Alternatively, in this step, when the metadata server detects whether the target file has been locked based on the ID of the target file, it can determine whether specific characters are recorded in a specific field in the directory of the target file based on the ID of the target file. If specific characters are recorded in the specific field in the directory of the target file, it can be determined that the target file has been locked. Alternatively, if specific characters are not recorded in the specific field in the directory of the target file, it can be determined that the target file is not locked.
[0075] In step S107, when the target file is not locked, the metadata server stores in the persistent storage space: the correspondence between the ID of the file stream client, the first locking sequence number, and the ID of the target file.
[0076] The persistent storage space can be a Store Cluster, etc.
[0077] In step S108, the metadata server locks the target file according to the ID of the target file.
[0078] For example, the metadata server may store the ID of the target file in the lock list to indicate that the target file has been locked, or may record a specific character (used to indicate that the target file has been locked) in a specific field in the directory of the target file. The specific character may be a number "1" or "2", etc.
[0079] In step S109, the metadata server returns a locking response to the file stream client, where the locking response is used to indicate that the target file has been locked.
[0080] For example, the lock response is used to indicate that the target file has been locked by the metadata server.
[0081] In step S110, the file stream client receives a lock response returned by the metadata server.
[0082] In step S111, the file stream client stores information indicating that the target file has been locked according to the locking response.
[0083] For example, the file stream client stores information indicating that the target file has been locked in the file stream client, for example, stores information indicating that the target file has been locked by the metadata server.
[0084] For example, the ID of the target file is recorded in a specific list in the file stream client to implement storage of information indicating that the target file has been locked.
[0085] In another embodiment of the present application, the write request also carries an operation mode for performing write operations on the target file, including add / delete / modify and incremental changes (for example, what content to add, delete or modify in the target file).
[0086] The lock response can also carry the directory of the target file in the Store Cluster. In this way, the file stream client can extract the directory of the target file in the Store Cluster from the lock response, and send a write instruction to the Store Cluster via the Chunk Client based on the directory of the target file in the Store Cluster. The write instruction carries the directory of the target file in the Store Cluster and the operation method for writing the target file. Secondly, it can also carry the ID of the target file.
[0087] In this way, the Store Cluster searches for the target file in the Store Cluster according to the directory of the target file in the write instruction, and performs a write operation on the target file in the manner specified in the write instruction.
[0088] Secondly, in another embodiment of the present application, if the file stream client completes the write operation on the target file, the file stream client needs to notify the metadata server to unlock the target file.
[0089] For example, the file stream client generates an unlock request, which carries the ID of the file stream client, the first locking sequence number, and the ID of the target file, and sends the unlock request to the metadata server.
[0090] If the metadata server determines that the file stream client ID in the unlock request is the same as the file stream client ID in the above correspondence and the first lock sequence number in the unlock request is the same as the first lock sequence number in the above correspondence, the target file is unlocked according to the ID of the target file.
[0091] For example, the metadata server may delete the ID of the target file from the lock list to indicate that the target file has been unlocked, or may delete specific characters (used to indicate that the target file has been unlocked) in a specific field in the directory of the target file.
[0092] In the present application, between "step S101 the file stream client obtains the first write request for the target file submitted by the user" to "step S108 when the target file is not locked, the metadata server stores in the persistent storage space: the correspondence between the ID of the file stream client, the first locking sequence number and the ID of the target file", if the metadata server fails (or crashes), the metadata server will not lock the target file, and the file stream client will not receive the locking response sent by the metadata server to indicate that the target file has been locked, and further the file stream client will not store the information indicating that the target file has been locked.
[0093] In this way, since the file stream client did not receive the locking response returned by the metadata server, the file stream client will believe that the target file is not locked. Even if other normal metadata servers subsequently replace the downed metadata server, since there is no "correspondence between the file stream client's ID, the first locking sequence number and the target file's ID" in the persistent storage space, other normal metadata servers will believe that the target file is not locked. It can be seen that the file stream client and other normal metadata servers will believe that the target file is not locked, so that the file stream client and other normal metadata servers have the same understanding of whether the target file is locked, avoiding deadlock.
[0094] In another embodiment of the present application, sometimes, the file stream client crashes before the metadata server returns a locking response to the file stream client, resulting in that even if the metadata server subsequently returns a locking response to the file stream client, the file stream client will definitely not receive the locking response sent by the metadata server, or, before the metadata server returns the locking response to the file stream client, the metadata server discovers that the communication connection between the metadata server and the file stream client is disconnected (possibly due to a network failure), resulting in that even if the metadata server subsequently sends a locking response to the file stream client, the file stream client will definitely not receive the locking response sent by the metadata server.
[0095] To this end, before the metadata server returns a locking response to the file stream client, if it is determined that the file stream client cannot receive the locking response, the metadata server may have locked the target file or may not have locked the target file yet. In this way, the metadata server can search for the corresponding relationship in the persistent storage space according to the ID of the file stream client. For example, in the persistent storage space, it searches for the corresponding relationship with the ID of the file stream client, etc., and determines whether the metadata server has locked the target file according to the ID of the target file in the corresponding relationship (the determination method can be seen in step S107, which is not described in detail here).
[0096] In the present application, it can be determined that the file stream client cannot receive the locking response in the following manner.
[0097] For example, if the metadata server detects that the communication connection between it and the file stream client is disconnected, it can be determined that the file stream client cannot receive the locking response.
[0098] Alternatively, the metadata server and the file stream client usually maintain a heartbeat. If the metadata server detects that the file stream client no longer has a heartbeat, it can be preliminarily determined that the file stream client is down, and it can be determined that the file stream client cannot receive a locking response.
[0099] Alternatively, the file stream controller and the file stream client usually maintain a heartbeat. When the file stream controller detects that the file stream client no longer has a heartbeat, it can preliminarily determine that the file stream client is down, and can then notify the metadata server of the file stream client downtime. In this way, the metadata server can determine, based on the notification, that the file stream client cannot receive a locking response.
[0100] When the metadata server has locked the target file, the target file is unlocked according to the ID of the target file.
[0101] In this way, since the file stream client does not receive the locking response returned by the metadata server, the file stream client will think that the target file is not locked, and the metadata server unlocks the target file according to the ID of the target file, and the metadata server will also think that the target file is not locked. It can be seen that the file stream client and the metadata server will both think that the target file is not locked, so that the file stream client and the metadata server have the same understanding of whether the target file is locked, avoiding deadlock. Through the embodiment of the present application, when the file stream client thinks that the target file is not locked, the metadata server can unlock the target file in time according to the ID of the target file, so that when other subjects need to perform write operations on the target file later, the metadata server can lock the target file for other subjects.
[0102] Alternatively, if the metadata server does not lock the target file, the process ends.
[0103] In another embodiment of the present application, sometimes due to a network failure, the locking response sent by the metadata server to the file stream client may be lost during the transmission process and cannot be received by the file stream client.
[0104] In this case, the metadata server has locked the target file according to the ID of the target file, but since the file stream client has not received a lock response, the file stream client will think that the target file has not been locked. This leads to inconsistent understanding of whether the target file is locked between the file stream client and the metadata server, which can easily lead to deadlock.
[0105] To this end, if the file stream client does not receive a lock response returned by the metadata server within a preset time after sending the first lock request to the metadata server, the file stream client can generate an unlock request for the target file, which carries the ID of the file stream client, the first lock sequence number and the ID of the target file, and sends the unlock request to the metadata server.
[0106] The preset duration may include 3 seconds, 4 seconds or 5 seconds, etc., which may be determined according to actual conditions and is not limited in this application.
[0107] The metadata server receives the unlock request sent by the file stream client, and determines whether the metadata server has locked the target file according to the ID of the target file.
[0108] If the metadata server does not lock the target file, the process ends.
[0109] Alternatively, when the metadata server has locked the target file, if the file stream client ID in the unlock request is the same as the file stream client ID in the corresponding relationship and the first lock sequence number in the unlock request is the same as the first lock sequence number in the corresponding relationship, the target file is unlocked according to the ID of the target file.
[0110] In this way, since the file stream client has not received the locking response returned by the metadata server, the file stream client will believe that the target file is not locked, and the metadata server will unlock the target file according to the ID of the target file, and the metadata server will also believe that the target file is not locked. It can be seen that both the file stream client and the metadata server will believe that the target file is not locked, so that the file stream client and the metadata server have the same understanding of whether the target file is locked, avoiding deadlock.
[0111] Through the embodiments of the present application, when the file stream client believes that the target file is not locked, the metadata server can promptly unlock the target file according to the ID of the target file, so that when other entities need to perform write operations on the target file later, the metadata server can lock the target file for other entities.
[0112] Alternatively, when the metadata server has locked the target file, if the file stream client ID in the unlock request is different from the file stream client ID in the corresponding relationship or the lock sequence number in the unlock request is different from the lock sequence number in the corresponding relationship, the target file will not be unlocked according to the target file ID.
[0113] In another implementation of the present application, due to the situation described in the above embodiment: the file stream client cannot receive the lock response, and thus cannot consider that the target file is locked, and thus cannot write to the target file in the Store Cluster. In this way, the user using the application subjectively perceives that he has been unable to write to the target file in the Store Cluster. In this case, sometimes the user may think that the first write request submitted previously has failed. In order to successfully write to the target file in the Store Cluster, the user may continue to submit a second write request to the target file through the application. The second write request carries at least the ID of the target file.
[0114] The file stream client obtains a second write request for the target file submitted by the user, then obtains a second lock sequence number based on the second write request, generates a second lock request for the target file, the second lock request carries the ID of the file stream client, the second lock sequence number and the ID of the target file, and sends the second lock request for the target file to the metadata server.
[0115] Before the metadata server receives the unlock request sent by the file stream client, if it receives a second lock request for the target file sent by the file stream client, since the metadata server locks the target file according to the ID of the target file in step S109, for the metadata server, the target file is in a locked state, and the ID of the target file in the corresponding relationship stored in the persistent storage space in step S108 is the same as the ID of the target file in the second lock request, and the ID of the file stream client in the corresponding relationship stored in the persistent storage space in step S108 is the same as the ID of the file stream client in the second lock request, and the first serial number in the corresponding relationship can be replaced by the second serial number in the persistent storage space.
[0116] Even if the metadata server receives an unlock request later, the metadata server will not unlock the target file according to the ID of the target file because the second lock sequence number in the corresponding relationship stored in the persistent storage space in step S108 is different from the first lock sequence number in the unlock request, thereby avoiding unlocking the target file when it should not be unlocked.
[0117] Furthermore, the metadata server may send a locking response indicating that the target file has been locked to the file stream client, so that the file stream client receives the locking response and stores information indicating that the target file has been locked according to the locking response.
[0118] In this application, the metadata server allows the same file to be read by multiple file stream clients simultaneously, but does not allow the same file to be written by multiple file stream clients simultaneously. Only one file stream client is allowed to write to the same file within a period of time or at any time.
[0119] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by the present application.
[0120] Reference Figure 4 , shows a file lock processing device of the present application, which is applied to a file stream client, and the device includes:
[0121] A first acquisition module 11 is used to acquire a first write request for a target file submitted by a user, where the first write request at least carries an ID of the target file;
[0122] A second acquisition module 12, configured to acquire a first locking sequence number according to the first write request;
[0123] A first generating module 13 is used to generate a first locking request for a target file, wherein the first locking request carries an ID of a file stream client, a first locking sequence number, and an ID of the target file;
[0124] The first sending module 14 is used to send a first locking request to the metadata server; so that the metadata server receives the first locking request for the target file sent by the file stream client; detects whether the target file has been locked according to the ID of the target file; if the target file is not locked, stores in the persistent storage space: the corresponding relationship between the ID of the file stream client, the first locking sequence number and the ID of the target file; locks the target file according to the ID of the target file; and returns a locking response to the file stream client, where the locking response is used to indicate that the target file has been locked;
[0125] The first receiving module 15 is used to receive a locking response returned by the metadata server;
[0126] The first storage module 16 is used to store information indicating that the target file has been locked according to the locking response.
[0127] Furthermore, the device also includes:
[0128] A second generating module is used to generate an unlocking request for the target file if no locking response is received from the metadata server within a preset time after the first locking request is sent to the metadata server, wherein the unlocking request carries the ID of the file stream client, the first locking sequence number, and the ID of the target file;
[0129] The second sending module is used to send an unlocking request to the metadata server, so that the metadata server determines whether the metadata server has locked the target file according to the ID of the target file; when the metadata server has locked the target file, if the file stream client ID in the unlocking request is the same as the file stream client ID in the corresponding relationship and the first locking sequence number in the unlocking request is the same as the first locking sequence number in the corresponding relationship, the target file is unlocked according to the ID of the target file.
[0130] Furthermore, the device also includes:
[0131] A third acquisition module is used to acquire a second write request for a target file submitted by a user; the second write request at least carries an ID of the target file;
[0132] A third generating module is used to obtain a second locking sequence number according to the second write request, and generate a second locking request for the target file, wherein the second locking request carries the ID of the file stream client, the second locking sequence number, and the ID of the target file;
[0133] The third sending module is used to send a second locking request for the target file to the metadata server, so that before the metadata server receives the unlocking request sent by the file stream client, if the metadata server receives the second locking request for the target file sent by the file stream client, the second serial number is used to replace the first serial number in the corresponding relationship in the persistent storage space.
[0134] In the present application, between "the file stream client obtains the first write request for the target file submitted by the user" to "when the target file is not locked, the metadata server stores in the persistent storage space: the correspondence between the ID of the file stream client, the first locking sequence number and the ID of the target file", if the metadata server fails (or crashes), the metadata server will not lock the target file, and the file stream client will not receive the locking response sent by the metadata server to indicate that the target file has been locked, and further the file stream client will not store the information indicating that the target file has been locked.
[0135] In this way, since the file stream client did not receive the locking response returned by the metadata server, the file stream client will believe that the target file is not locked. Even if other normal metadata servers subsequently replace the downed metadata server, since there is no "correspondence between the file stream client's ID, the first locking sequence number and the target file's ID" in the persistent storage space, other normal metadata servers will believe that the target file is not locked. It can be seen that the file stream client and other normal metadata servers will believe that the target file is not locked, so that the file stream client and other normal metadata servers have the same understanding of whether the target file is locked, avoiding deadlock.
[0136] Reference Figure 5 , shows a file lock processing device of the present application, which is applied to a metadata server, and the device includes:
[0137] The second receiving module 21 is used to receive a first lock request for a target file sent by a file stream client, the first lock request carries the ID of the file stream client, the first lock sequence number and the ID of the target file; the first lock sequence number is obtained by the file stream client according to a first write request for the target file submitted by a user, the first write request carries at least the ID of the target file; the first lock request is generated by the file stream client;
[0138] A detection module 22, used to detect whether the target file has been locked according to the ID of the target file;
[0139] The second storage module 23 is used to store in the persistent storage space: the correspondence between the ID of the file stream client, the first locking sequence number and the ID of the target file when the target file is not locked;
[0140] A locking module 24, used to lock the target file according to the ID of the target file;
[0141] The return module 25 is used to return a locking response to the file stream client, where the locking response is used to indicate that the target file has been locked, so that the file stream client stores information indicating that the target file has been locked according to the locking response.
[0142] Furthermore, the device also includes:
[0143] A third receiving module is used to receive an unlocking request for a target file sent by a file stream client, wherein the unlocking request carries an ID of the file stream client, a first locking sequence number, and an ID of the target file, and the unlocking request is generated and sent to the metadata server when the file stream client does not receive a locking response returned by the metadata server within a preset time after sending the first locking request to the metadata server;
[0144] A determination module, used to determine whether the metadata server has locked the target file according to the ID of the target file;
[0145] The unlocking module is used to unlock the target file according to the ID of the target file when the metadata server has locked the target file, if the file stream client ID in the unlocking request is the same as the file stream client ID in the corresponding relationship and the first locking sequence number in the unlocking request is the same as the first locking sequence number in the corresponding relationship.
[0146] Furthermore, the device also includes:
[0147] A replacement module is used to replace the first serial number in the corresponding relationship in the persistent storage space with a second serial number if a second lock request for a target file is received from the file stream client before the metadata server receives an unlock request from the file stream client. The second lock request carries the ID of the file stream client, the second lock serial number and the ID of the target file. The second lock serial number is obtained by the file stream client based on a second write request for the target file submitted by the user, and the second write request carries at least the ID of the target file. The second lock request is generated by the file stream client.
[0148] In the present application, between "the file stream client obtains the first write request for the target file submitted by the user" to "when the target file is not locked, the metadata server stores in the persistent storage space: the correspondence between the ID of the file stream client, the first locking sequence number and the ID of the target file", if the metadata server fails (or crashes), the metadata server will not lock the target file, and the file stream client will not receive the locking response sent by the metadata server to indicate that the target file has been locked, and further the file stream client will not store the information indicating that the target file has been locked.
[0149] In this way, since the file stream client did not receive the locking response returned by the metadata server, the file stream client will believe that the target file is not locked. Even if other normal metadata servers subsequently replace the downed metadata server, since there is no "correspondence between the file stream client's ID, the first locking sequence number and the target file's ID" in the persistent storage space, other normal metadata servers will believe that the target file is not locked. It can be seen that the file stream client and other normal metadata servers will believe that the target file is not locked, so that the file stream client and other normal metadata servers have the same understanding of whether the target file is locked, avoiding deadlock.
[0150] Optionally, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0151] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0152] Figure 6 800 is a block diagram of an electronic device 800 shown in the present application. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0153] Reference Figure 6, the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .
[0154] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.
[0155] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, images, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0156] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.
[0157] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also monitor the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0158] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.
[0159] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.
[0160] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can monitor the open / closed state of the device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also monitor the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to monitor the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0161] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast operation information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0162] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.
[0163] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0164] Figure 7 1 is a block diagram of an electronic device 1900 shown in the present application. For example, the electronic device 1900 may be provided as a server.
[0165] Reference Figure 7 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.
[0166] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.
[0167] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0169] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
[0170] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0171] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0172] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0173] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0174] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0175] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.
[0176] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for processing a file lock, characterized in that: Applied to a file stream client, the method comprises: Obtaining a first write request for a target file submitted by a user, where the first write request carries at least an identity identifier ID of the target file; Obtain a first lock sequence number according to the first write request; Generate a first lock request for the target file, the first lock request carrying the ID of the file stream client, the first lock sequence number, and the ID of the target file; Sending a first lock request to a metadata server; allowing the metadata server to receive the first lock request for a target file sent by a file stream client; detecting whether the target file has been locked according to the ID of the target file; storing in a persistent storage space: a correspondence between the ID of the file stream client, the first lock sequence number, and the ID of the target file if the target file has not been locked; locking the target file according to the ID of the target file; and returning a lock response to the file stream client, the lock response being used to indicate that the target file has been locked; Receive the lock response returned by the metadata server; Information indicating that the target file has been locked is stored according to the lock response.
2. The method according to claim 1, characterized in that: The method further comprises: If no locking response is received from the metadata server within a preset time after the first locking request is sent to the metadata server, an unlocking request for the target file is generated, where the unlocking request carries the ID of the file stream client, the first locking sequence number, and the ID of the target file; An unlock request is sent to the metadata server, so that the metadata server determines whether the metadata server has locked the target file according to the ID of the target file; if the metadata server has locked the target file, if the file stream client ID in the unlock request is the same as the file stream client ID in the corresponding relationship and the first lock sequence number in the unlock request is the same as the first lock sequence number in the corresponding relationship, the target file is unlocked according to the ID of the target file.
3. The method according to claim 2, characterized in that The method further comprises: Obtaining a second write request for the target file submitted by the user; the second write request carries at least the ID of the target file; Obtain a second lock sequence number according to the second write request; Generate a second lock request for the target file, the second lock request carrying the ID of the file stream client, the second lock sequence number, and the ID of the target file; A second lock request for the target file is sent to the metadata server, so that before the metadata server receives the unlock request sent by the file stream client, if the metadata server receives the second lock request for the target file sent by the file stream client, the second serial number is used to replace the first serial number in the corresponding relationship in the persistent storage space.
4. A method for processing a file lock, characterized in that: Applied to a metadata server, the method includes: Receive a first lock request for a target file sent by a file stream client, the first lock request carrying an identity identifier ID of the file stream client, a first lock sequence number, and an ID of the target file; the first lock sequence number is obtained by the file stream client according to a first write request for the target file submitted by a user, the first write request carrying at least the ID of the target file; the first lock request is generated by the file stream client; According to the ID of the target file, check whether the target file has been locked; When the target file is not locked, the corresponding relationship between the ID of the file stream client, the first locking sequence number and the ID of the target file is stored in the persistent storage space; Lock the target file according to its ID; A locking response is returned to the file stream client, where the locking response is used to indicate that the target file has been locked, so that the file stream client stores information indicating that the target file has been locked according to the locking response.
5. The method according to claim 4, characterized in that The method further comprises: Receive an unlock request for a target file sent by a file stream client, the unlock request carrying an ID of the file stream client, a first lock sequence number, and an ID of the target file, the unlock request being generated and sent to the metadata server when the file stream client does not receive a lock response returned by the metadata server within a preset time after sending the first lock request to the metadata server; Determine whether the metadata server has locked the target file according to the ID of the target file; When the metadata server has locked the target file, if the file stream client ID in the unlock request is the same as the file stream client ID in the corresponding relationship and the first lock sequence number in the unlock request is the same as the first lock sequence number in the corresponding relationship, the target file is unlocked according to the ID of the target file.
6. The method according to claim 5, characterized in that The method further comprises: Before the metadata server receives the unlocking request sent by the file stream client, if it receives a second locking request for the target file sent by the file stream client, in the persistent storage space, the first serial number in the corresponding relationship is replaced by the second serial number; the second locking request carries the ID of the file stream client, the second locking serial number and the ID of the target file; the second locking serial number is obtained by the file stream client based on the second write request for the target file submitted by the user, and the second write request carries at least the ID of the target file; the second locking request is generated by the file stream client.
7. A file lock processing device, characterized in that: Applied to a file stream client, the device comprises: A first acquisition module, used to acquire a first write request for a target file submitted by a user, wherein the first write request carries at least an identity identifier ID of the target file; A second acquisition module, used to acquire a first locking sequence number according to the first write request; A first generating module, used to generate a first locking request for a target file, wherein the first locking request carries an ID of a file stream client, a first locking sequence number, and an ID of the target file; The first sending module is used to send a first locking request to the metadata server; so that the metadata server receives the first locking request for the target file sent by the file stream client; detects whether the target file has been locked according to the ID of the target file; if the target file is not locked, stores in the persistent storage space: a correspondence between the ID of the file stream client, the first locking sequence number and the ID of the target file; locks the target file according to the ID of the target file; and returns a locking response to the file stream client, where the locking response is used to indicate that the target file has been locked; A first receiving module, used to receive a lock response returned by the metadata server; The first storage module is used to store information indicating that the target file has been locked according to the locking response.
8. A file lock processing device, characterized in that: Applied to a metadata server, the device comprises: A second receiving module is used to receive a first lock request for a target file sent by a file stream client, wherein the first lock request carries an identity identifier ID of the file stream client, a first lock sequence number, and an ID of the target file; the first lock sequence number is obtained by the file stream client according to a first write request for the target file submitted by a user, wherein the first write request carries at least the ID of the target file; the first lock request is generated by the file stream client; A detection module, used to detect whether the target file has been locked according to the ID of the target file; The second storage module is used to store, in the persistent storage space, the corresponding relationship between the ID of the file stream client, the first locking sequence number and the ID of the target file when the target file is not locked; A locking module is used to lock the target file according to the ID of the target file; The return module is used to return a locking response to the file stream client, where the locking response is used to indicate that the target file has been locked, so that the file stream client stores information indicating that the target file has been locked according to the locking response.
9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the file lock processing method according to any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the file lock processing method according to any one of claims 1 to 6 is implemented.