Disk file read-write method and device, electronic equipment and storage medium

Abstract

The invention discloses a read-write method and device of a disk file, electronic equipment and a storage medium, and relates to the technical field of computers.The method comprises the steps that under the condition that a forwarding mark of a to-be-read-write file is stored in latest metadata information of the to-be-read-write file, a read-write forwarding message is constructed; the forwarding mark is cached to the latest metadata information under the condition that a plurality of clients simultaneously initiate metadata opening requests for the to-be-read-write file, and the forwarding mark represents that the read-write request for the to-be-read-write file is forwarded to the target centralized control client; the target centralized control client is a client which uniformly processes concurrent read-write requests of a plurality of clients for the file to be read and written; and forwarding the read-write forwarding message to the target centralized control client so as to perform read-write operation on the file to be read and written in the disk through the target centralized control client. Through the method and the device, the problems of slow system response time and small throughput caused by large interaction pressure between metadata service and multiple clients in the related technology are solved.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and in particular, to a method and apparatus for reading and writing disk files, an electronic device, and a storage medium. Background Art

[0002] Ceph is a unified distributed storage system that provides block storage services, object storage services, and file system storage services at the same time. The Metadata Services (MDS) is the metadata service of the Ceph file system, which is used to process the metadata of the file system storage service and coordinate the access to the storage cluster. The Object Storage Device (OSD) is a component of Ceph, which is mainly responsible for the daemon process that responds to client requests and returns specific data. Generally, a Ceph cluster will have multiple OSDs, and each disk will correspond to an OSD.

[0003] In large-scale data centers and cloud computing environments, Ceph is widely used in high-concurrency scenarios, such as large-scale video streams, big data analysis, and cloud storage services, due to its powerful data storage and management capabilities. Traditionally, Ceph maintains data consistency by MDS permission control and OSD centralized control of the write order. However, when using other storage backends to replace OSDs, the centralized control function of OSDs is lost, and concurrent read and write of the same file by multiple clients may lead to data inconsistency.

[0004] In related technologies, in the case of concurrent read and write by multiple clients, MDS realizes direct read and write through permission control. Although it ensures basic data consistency, MDS needs to communicate with all clients frequently for permission granting and revocation messages, and needs to perform frequent lock state switches, significantly increasing the computing pressure on MDS, making it a performance bottleneck in high-concurrency scenarios and affecting the response time and throughput of the entire system. Summary of the Invention

[0005] This application provides a method and apparatus for reading and writing disk files, an electronic device, and a storage medium, so as to at least solve the problem in related technologies that the interaction pressure between the metadata service and multiple clients is large, resulting in slow system response time and small throughput.

[0006] The present application provides a method for reading and writing disk files, which is applied to a client and includes: obtaining the latest metadata information of the file to be read and written from the local cache; constructing a read / write forwarding message and determining a target centralized control client when a forwarding flag of the file to be read and written is stored in the latest metadata information; the forwarding flag is cached in the latest metadata information when multiple clients simultaneously initiate an open metadata request for the file to be read and written, and the forwarding flag indicates that the read / write request for the file to be read and written is forwarded to the target centralized control client; the target centralized control client refers to the client that uniformly processes the concurrent read / write requests of the multiple clients for the file to be read and written; forwarding the read / write forwarding message to the target centralized control client to perform read / write operations on the file to be read and written in the disk through the target centralized control client.

[0007] The present application further provides a device for reading and writing disk files, which is applied to a client and includes: a first execution unit for obtaining the latest metadata information of the file to be read and written from the local cache; a second execution unit for constructing a read / write forwarding message and determining a target centralized control client when a forwarding flag of the file to be read and written is stored in the latest metadata information; the forwarding flag is cached in the latest metadata information when multiple clients simultaneously initiate an open metadata request for the file to be read and written, and the forwarding flag indicates that the read / write request for the file to be read and written is forwarded to the target centralized control client; the target centralized control client refers to the client that uniformly processes the concurrent read / write requests of the multiple clients for the file to be read and written; a third execution unit for forwarding the read / write forwarding message to the target centralized control client to perform read / write operations on the file to be read and written in the disk through the target centralized control client.

[0008] The present application further provides an electronic device, including: a memory for storing a computer program; a processor for implementing any of the above methods when executing the computer program.

[0009] The present application further provides a computer-readable storage medium, in which a computer program is stored, and the computer program implements any of the above methods when executed by a processor.

[0010] Through this application, when multiple clients initiate open metadata requests for the same file to be read and written, a forwarding flag of the file to be read and written is stored in the metadata information to obtain the latest metadata information. The forwarding flag is used to indicate that when the client performs subsequent read and write operations, the read and write requests for the file to be read and written are forwarded to the target centralized control client; the forwarding flag is set during permission update so as to determine whether read and write forwarding is required according to this forwarding flag when performing read and write operations, without frequent permission granting and revocation message communication with the metadata service, and without frequent lock state switching, reducing the interaction pressure between the metadata service and the client. Moreover, by performing read and write operations on the file to be read and written through the target centralized control client, data chaos caused by concurrent operations is eliminated, so that read and write requests of all clients can be processed at the target centralized control client, avoiding multiple clients writing to the disk simultaneously, reducing frequent interactions between multiple clients and the metadata service, alleviating the load of the metadata service, and improving the response time and throughput of the system. Therefore, the problem of slow system response time and small throughput caused by large interaction pressure between the metadata service and multiple clients in the related art can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] To more clearly illustrate the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 FIG. is a schematic diagram of an application scenario of a method for reading and writing a disk file according to an embodiment of the present application.

[0013] Figure 2 FIG. is a schematic flowchart of a method for reading and writing a disk file according to an embodiment of the present application.

[0014] Figure 3 FIG. is a flowchart framework diagram of an optional method for reading and writing a disk file according to an embodiment of the present application.

[0015] Figure 4 FIG. is a timing diagram of an optional method for reading and writing a disk file according to an embodiment of the present application.

[0016] Figure 5 FIG. is a structural block diagram of an optional device for reading and writing a disk file according to an embodiment of the present application.

[0017] Figure 6 FIG. is a structural block diagram of a computer system of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0019] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0020] In order to enable those skilled in the art of the present technology to better understand the solutions of the present application, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0021] According to one aspect of the embodiments of the present application, a method for reading and writing disk files is provided. Optionally, in this embodiment, the above method for reading and writing disk files may be but is not limited to being applied to, for example, Figure 1 as shown, including multiple clients 102, a metadata server 104, and a file storage server 106. The multiple clients 102 communicate with the metadata server 104 and the file storage server 106 respectively, and can be used to provide services (such as application services, etc.) for the multiple clients 102 installed on the terminal device. The metadata server 104 is responsible for managing and coordinating the metadata of the file system, including information such as file location, permissions, and attributes. The file storage server 106 is responsible for storing the actual file data, and through cooperation with the metadata server, efficiently responds to read and write requests to meet the needs of large-scale data access. A database can be set on the metadata server 104 and the file storage server 106 or independently of the metadata server 104 and the file storage server 106 to provide data storage services for the metadata server 104 and the file storage server 106.

[0022] The above network may include, but is not limited to, at least one of the following: a wired network, a wireless network. The above wired network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, a local area network. The above wireless network may include, but is not limited to, at least one of the following: Wireless Fidelity (WIFI), Bluetooth. The client 102 may be, but is not limited to, a personal computer (PC), a mobile phone, a tablet computer, etc. The metadata server 104 and the file storage server 106 may be, but are not limited to, cloud servers, server clusters, or other server types.

[0023] The read and write method of the disk file in the embodiment of the present application may be executed by the client 102. Taking the execution of the read and write method of the disk file in this embodiment by the client as an example, Figure 2 It is a schematic flowchart of an optional read and write method of a disk file according to an embodiment of the present application, as Figure 2 shown, the process of this method may include steps S202 to S206.

[0024] Step S202, obtain the latest metadata information of the file to be read and written from the local cache.

[0025] Step S204, when a forwarding flag of the file to be read and written is stored in the latest metadata information, construct a read and write forwarding message and determine the target centralized control client; the forwarding flag is cached in the latest metadata information when multiple clients simultaneously initiate an open metadata request for the file to be read and written, and the forwarding flag indicates that the read and write request for the file to be read and written is forwarded to the target centralized control client; the target centralized control client refers to the client that uniformly processes the concurrent read and write requests of multiple clients for the file to be read and written.

[0026] Step S206, forward the read and write forwarding message to the target centralized control client, so as to perform read and write operations on the file to be read and written in the disk through the target centralized control client.

[0027] The read-write method of disk files in the embodiments of this application can be applied to the technical field of distributed storage systems and applied to high-concurrency scenarios (for example, large-scale video streams, big data analysis, and cloud storage services). Ceph is a unified distributed storage system that provides block storage services, object storage services, and file system storage services at the same time. The Meta Data Services (MDS for short) is the metadata service of the Ceph file system, which is used to process the metadata of the file system storage service and coordinate the access to the storage cluster. The client is the interface that interacts with users in the Ceph system, responsible for accessing and operating the data in the Ceph storage cluster, and implements data access, request forwarding, and interaction interfaces. The Object Storage Device (OSD for short) is a component of Ceph, which is mainly responsible for the daemon process that responds to client requests and returns specific data. Generally, a Ceph cluster will have multiple OSDs, and each disk will correspond to an OSD.

[0028] In the deployment scenario where other storage backends are used to replace the OSD, for the situation where multiple clients read and write simultaneously, the related technology realizes the control of the read-write order of the clients through MDS permission control. In the scenario where the MDS determines that multiple clients open a file simultaneously, according to the order in which each client applies for the write permission, the client is successively granted exclusive read-write permissions, and other clients wait serially.

[0029] However, in the above related technology, in the case of multiple clients reading and writing simultaneously, the MDS realizes direct reading and writing through permission control. Although it ensures basic data consistency, the MDS needs to communicate frequently with all clients for permission granting and revocation messages, and needs to perform frequent lock state switches, which significantly increases the computing pressure of the MDS and makes it a performance bottleneck in high-concurrency scenarios, affecting the response time and throughput of the entire system.

[0030] In order to at least partially solve the above technical problems, in this embodiment, when multiple clients initiate open metadata requests for the same file to be read / written, a forwarding flag of the file to be read / written is stored in the metadata information, a read / write forwarding message is constructed, and a target centralized control client is determined. When the permissions are updated, the forwarding flag is set so as to decide whether read / write forwarding is required based on this forwarding flag when performing read / write operations, and the read / write forwarding message is sent to the target centralized control client. After receiving the read / write forwarding message, the target centralized control client processes the read / write operations, eliminating the data chaos that may be caused by concurrent operations, so that the read / write requests of all clients can be processed by the target centralized control client, avoiding multiple clients writing to the disk simultaneously, reducing the frequent interaction between multiple clients and the metadata service, and reducing the load on the metadata service. Therefore, the problem in the related art that the MDS needs to communicate with all clients for frequent permission granting and revocation messages and needs to perform frequent lock state switches, affecting the response time and throughput of the entire system, can be solved.

[0031] When a client actively opens a file to be read / written in a read / write manner, it will send an open metadata request to the MDS. This open data request is used to initialize the access to the file to be read / written. After the open metadata request is successful, the client can start subsequent actual read / write operations. When the MDS receives the open data request, it will check the access status of the file to be read / written. In the case where multiple clients simultaneously initiate open metadata requests for the file to be read / written, the MDS will inform the client of the access status of the file to be read / written and the metadata information of the file to be read / written. In the case where the access status of the file to be read / written indicates that multiple clients simultaneously initiate open metadata requests for the file to be read / written, the client sets a forwarding flag in the metadata information of the file to be read / written. The above forwarding flag can be a specific identifier used to indicate that a certain file is in a state of concurrent access by multiple clients and to indicate that read / write requests for the file to be read / written are forwarded to the target centralized control client.

[0032] The above metadata information can be regarded as a set of attributes of the file to be read / written, including the inode of the file (the inode is a data structure in the file system used to store metadata about a file or directory, including information such as the size, type, permissions, owner, modification time, etc. of the file), the permission status, and other metadata information, which is stored in the local cache of the client. The latest metadata information can ensure that the client can obtain the latest status of the file when performing read / write operations. In the case where the forwarding flag of the file to be read / written is stored in the latest metadata information, all read / write operations for this file will be processed by the target centralized control client instead of being independently performed by each client.

[0033] The above-mentioned constructing read / write forwarding message is a step taken by the client (initiating client) after detecting the forwarding flag, and is used to perform read / write operations on the file to be read / written. The read / write forwarding message may include, but is not limited to, the inode information of the file, the operation type (read or write), the offset, the operation length, and the data content to be read / written.

[0034] The target centralized control client may refer to a client set up in a distributed file system architecture to handle concurrent read / write requests from multiple clients for the same file. The target centralized control server can be used to receive read / write forwarding messages from other clients (e.g., Initiator clients), and perform read / write operations on the file on the underlying disk or storage device according to the instructions and order in the read / write forwarding messages. The target centralized control server can be an additional client added in the distributed file system architecture besides multiple clients.

[0035] The above-mentioned method for determining the target centralized control client can be to use the Hash algorithm, taking the identification information of the file to be read / written (such as file name, inode number, etc.) as input, generating a fixed Hash value, and then selecting a centralized control client from a group of preset centralized control clients as the target centralized control client according to this Hash value. The method for determining the target centralized control client can also be to use the polling mechanism to select the target centralized control client, that is, each time a read / write forwarding request is received, the next node in the centralized control client list is selected cyclically in a certain order. The method for determining the target centralized control client can also be determined through the inode information and the MDS. The client calculates the belonging MDS (target metadata service) of the file to be read / written through the inode information, and then obtains the communication address of the target centralized control client bound to this belonging MDS.

[0036] Send the read / write forwarding message to the target centralized control client. After receiving all the forwarding messages, the target centralized control client will perform read or write operations on the file to be read / written on the disk according to the arrival order of the messages. It should be noted that the method of performing read / write operations on the file to be read / written on the disk through the target centralized control client can be that after the target centralized control client receives read / write requests forwarded from different clients, it can perform serial processing according to the order of the requests, or the target centralized control client directly interacts with the disk to perform read / write operations on the file to be read / written. In the embodiments of the present application, the target centralized control client only processes data read / write on the disk and does not update metadata. After the read / write is completed, it sends a response message to the client. For a read forwarding request, it can reply with the result of the read data and the read data content; for a write forwarding request, it can reply with the length of the written data.

[0037] For example, the target centralized control client is bound to the MDS, that is, each MDS is bound to a target centralized control client (Target client). For the files belonging to this MDS, their forwarded messages are fixedly sent to the Target client for processing. This target centralized control client serves as the destination for read / write forwarding requests and is dedicated to processing read / write requests sent by other clients (initiator clients, i.e., Initiator clients). According to the request order, it serially completes the data read and write operations to the disk and returns the read / write execution results and the read data to the initiator client through the forwarded message. It can be understood that when multiple clients simultaneously attempt to read and write the same file, the MDS can reclaim the cache permissions of all clients and grant all clients the direct read and write permissions. After detecting the forwarding flag, the Initiator client forwards the read / write request to the Target client bound to the MDS to which the inode of the file belongs. The Target client is responsible for serially processing all read / write requests to ensure data consistency.

[0038] Optionally, the client opens a file in read / write mode. To this end, it obtains the latest metadata information of the file from its local cache. If the file is currently in a state of multi-client concurrent access, then a forwarding flag can be found in its metadata information. After discovering the forwarding flag, the client immediately constructs a read / write forwarding message, which includes the inode information of the file, the request type (such as read or write), the starting address offset, the operation length, and the relevant data content. Then, the client uses the inode information to determine the identity and communication address of the target centralized control client, and subsequently sends the read / write forwarding message to the target centralized control client. The target centralized control client receives the read / write requests for the files to be read and written sent by multiple clients, and the target centralized control client can process them in the order of the received read / write requests. When the target centralized control client completes the read / write operations on the files to be read and written in the disk, the target centralized control client converts the results and the read data (if it is a read operation) into a reply message and sends it back to the client that sent the read / write request, allowing the client to update its local metadata cache and complete the read / write operation.

[0039] For example, Figure 3 is a flowchart framework of an optional method for reading and writing disk files provided according to an embodiment of the present application. As Figure 3 shown, there are two clients, namely client A and client B. Client A sends a write forwarding message A1 to the target centralized control client, and client B sends a write forwarding message B1 to the target centralized control client. The target centralized control client sequentially writes data A2 and data B2 into file 1 in the disk. After completion, the target centralized control client sends a forwarding message reply A3 to client A and a forwarding message reply B3 to client B respectively.

[0040] According to the embodiments of the present application, when multiple clients initiate an open metadata request for the same file to be read and written, a forwarding flag of the file to be read and written is stored in the metadata information to obtain the latest metadata information, where the forwarding flag is used to indicate that when the client performs subsequent read and write operations, the read and write requests for the file to be read and written are forwarded to the target centralized control client; the forwarding flag is set during permission update so as to determine whether read and write forwarding is required according to this forwarding flag when performing read and write operations, without frequent permission granting and revocation message communication with the metadata service, and without frequent lock state switching, reducing the interaction pressure between the metadata service and the client, and performing read and write operations on the file to be read and written through the target centralized control client, eliminating data chaos that may be caused by concurrent operations, so that the read and write requests of all clients can be processed at the target centralized control client, avoiding multiple clients writing to the disk simultaneously, reducing the frequent interaction between multiple clients and the metadata service, and reducing the load on the metadata service. Therefore, the problem of large interaction pressure between the metadata service and multiple clients in the related art, resulting in slow system response time and small throughput, can be solved.

[0041] In some exemplary embodiments, before obtaining the latest metadata information of the file to be read and written from the local cache, the above method further includes steps one to three.

[0042] 1. Determine a target metadata service corresponding to the file to be read and written according to the file to be read and written; the target metadata service is used to manage the read and write permissions of the file to be read and written.

[0043] 2. Send an open metadata request to the target metadata service and receive a metadata response message fed back by the target metadata service; the metadata response message includes the original metadata information of the file to be read and written and file permission information.

[0044] 3. When the file permission information represents non-exclusive permissions, cache the forwarding flag in the original metadata information to obtain the latest metadata information.

[0045] The above determination of the target metadata service corresponding to the file to be read and written, that is, the client identifies the metadata service (MDS) responsible for managing the read and write permissions of the file to be read and written. Since in the Ceph distributed file system, the metadata information of each file may be responsible for by different MDSs, the client can determine the target MDS for managing the read and write permissions of the file to be read and written by calculating or querying the mapping relationship between the inode information of the file and the MDS.

[0046] Before the client needs to perform read and write operations on the file to be read and written, it can first send an open metadata request to the target MDS. The open metadata request includes the identification information of the file to be read and written. After receiving the request, the target MDS can check the current status of the file to be read and written, and then form a metadata response message and feedback it to the client. The metadata response message can refer to the response returned by the metadata service to the client after processing the open metadata request when the client sends the open metadata request to the metadata service. For example, the metadata response message can include the original metadata information and file permission information of the file to be read and written. Metadata information can refer to the data used to describe the attributes and structure of the file to be read and written. For example, the original metadata information can include, but is not limited to, the identification information of the file to be read and written, the attributes of the file to be read and written (such as the size, creation time, modification time, access permission, etc. of the file), the file structure information (such as the block distribution of the file, the storage location of the data block, and the relationship between the file and the directory, etc.), and the file permission information (the access control information related to the file, such as read permission, write permission, and execution permission, etc.). File permission information can refer to the data that controls the access level of the client to the file, usually including read permission, write permission, and execution permission, as well as whether the specific file (the file to be read and written) is set to the exclusive access state. File permission information is used to determine whether the client is an exclusive client, that is, to judge whether the client can directly read and write the file to be read and written, or whether it needs to forward the read and write forwarding information to the target centralized control client for serialization processing. For example, the file permission information is controlled and allocated by the target MDS. After receiving the file permission information through the metadata response message, the client can decide whether it can directly operate on the file to be read and written by itself, or whether it needs to forward the read and write forwarding request to the target centralized control client.

[0047] In the case where the file permission information represents exclusive permission, it means that the file to be read and written is only opened by one client in a read and write manner. Then this client has full operation permissions for the file to be read and written, such as reading cache / writing buffer / reading and writing data / modifying metadata, etc. In the case where the file permission information represents non-exclusive permission, it means that the current file to be read and written may be accessed by multiple clients simultaneously. Multiple clients only have the permissions of direct reading and direct writing. In this case, the client caches the forwarding flag in the original metadata information of the file to be read and written in the cache to obtain the latest metadata information. The forwarding flag can mark that all read and write requests for the same file are forwarded to the target centralized control client for processing.

[0048] For example, if the file to be read and written is only opened by one client in a read-write mode, then this client has full operation permissions for the file to be read and written (read cache / write buffer / read and write data / modify metadata, etc.). This client can be called an exclusive client. In this state, all read and write requests of the exclusive client are serially processed. When the file to be read and written is opened by multiple clients in a read-write mode at the same time, the target MDS can reclaim the cache permissions of all clients and grant the direct read and write permissions. At this time, there is no exclusive client. At this time, each client sets a forwarding flag in the file inode information in the metadata cache. Subsequently, all read and write requests of the client for the file to be read and written will trigger the logic of read and write forwarding.

[0049] When the file permission message indicates exclusivity, the client can directly obtain or update the metadata information of the file from the local cache and directly perform read and write operations on the file in the disk. For example, when the client receives a metadata response message and the file permission information in it indicates that the client has exclusive permissions for the file to be read and written, the client can perform the following operations: If a forwarding flag was previously set for the file to be read and written, the forwarding flag will be cleared at this time; the client updates the original metadata information in the local cache according to the received metadata response message; after the client confirms the permission status, it can directly perform read and write operations without constructing a forwarding message or determining the target centralized control client, reducing the latency and communication overhead on the read and write path.

[0050] Through the embodiments of the present application, when the client detects non-exclusive permissions, that is, when the file to be read and written is accessed by multiple clients at the same time, the client adds a forwarding flag to the metadata in the local cache, which is used to indicate that the client needs to forward its own read and write requests to a specific target client instead of directly operating on the data, avoiding data overwrite and inconsistency problems that may be caused by direct concurrent writing, and at the same time reducing the lock switching and permission management pressure of the MDS in a high-concurrency scenario. In addition, by saving the forwarding flag in the local metadata cache of the client instead of synchronizing it to the MDS, the number of interactions between the client and the MDS is reduced, thereby significantly reducing the communication overhead.

[0051] In some exemplary embodiments, the file to be read and written is assigned first identification information; determining the target metadata service corresponding to the file to be read and written according to the file to be read and written includes: looking up a preset first local mapping table according to the first identification information to obtain the target metadata service; the first local mapping table stores the mapping relationship between different files and metadata services.

[0052] In the embodiments of the present application, the first identification information may refer to information used to uniquely identify and locate the file or data object to be read and written. For example, the first identification information may refer to the inode number of the file to be read and written. The inode number may be an internal number used to uniquely identify each file in the Ceph file system, independent of the file name. Even if the file name changes, the inode number remains unchanged.

[0053] The above-mentioned first local mapping table may be a data structure preset locally on the client, used to store the mapping relationship between different files and the metadata service. In the Ceph distributed file system, in order to balance the load and improve efficiency, different files may be managed by different MDSs. The first local mapping table records this mapping relationship. By querying the first local mapping table, the client can quickly determine the target metadata service corresponding to the file to be read and written. Before the client needs to perform a read and write operation on a file to be read and written, it obtains the first identification information of the file to be read and written, and then searches in the first local mapping table locally on the client to determine the target metadata service that matches the file to be read and written.

[0054] For example, when reading and writing a file to be read and written, the first identification information of the file to be read and written, such as the inode number, can be obtained by calling the local cache. Subsequently, the client uses this inode number as the key value to query the preset first local mapping table, which is filled when the client is initialized and records the corresponding relationship between all files and the MDS responsible for their metadata management. What is returned by the query of the first local mapping table may be the ID of the target MDS associated with the file to be read and written. After obtaining the ID of the target MDS, the client identifies the target MDS that manages the read and write permissions of this file, and thus can send an open metadata request to the target MDS.

[0055] Through the embodiments of the present application, by introducing the first identification information and the preset first local mapping table, it is realized that before the client performs a file opening operation, it can quickly and accurately determine the target metadata service that manages the read and write permissions of the file to be read and written, reducing the frequency and latency of message communication, improving the response speed and concurrent processing ability of the system, and ensuring that when multiple clients concurrently read and write the same file, efficient and accurate metadata permission management can be carried out.

[0056] In some exemplary embodiments, the target metadata service is assigned with second identification information; the above-mentioned determining the target centralized control client includes: according to the second identification information, searching the preset second local mapping table to determine the target centralized control client; the second local mapping table stores the mapping relationship between different metadata services and different centralized control clients.

[0057] The above-mentioned second identification information may refer to the identification information of the target metadata service. For example, the second identification information may be the ID of the target metadata service or any symbol that uniquely identifies the target metadata service. The second identification information can be used to find the target centralized control client associated with the target metadata service in the local mapping table of the client. Among them, the identification information of the target metadata service can be set during the initialization or configuration phase.

[0058] The second local mapping table can be a data structure pre-set locally on the client, which is used to store the mapping relationships between different metadata services and different centralized control clients. The second local mapping table can ensure that in the case where multiple clients simultaneously initiate open metadata requests for files to be read and written, the read and write operation requests for the files to be read and written are forwarded to the correct target centralized control client for processing. By querying the second local mapping table, the client can quickly determine the target centralized control client bound to the target MDS to which the file to be read and written belongs.

[0059] In this embodiment, after the client confirms that the read and write messages for the file to be read and written need to be forwarded for processing, it can use the second identification information of the target metadata service to search in the second local mapping table. The search process can utilize data structures such as indexes or hash tables, or the Controlled, Scalable, Decentralized Placement Strategy (CRUSH) algorithm of Ceph or other similar mechanisms, ensuring the efficiency and accuracy of the search, thereby determining the target centralized control client bound to the target metadata service.

[0060] For example, after the forwarding message combination is completed, obtain the id of its target metadata service according to the inode number, and obtain the address of the Target client. After sending the message, wait for the message to return. The client has a second mapping table, which records the addresses of the Target clients associated with the ids of each MDS. The client calculates the id of the target MDS (the ID of the target metadata service) to which the file belongs according to the inode information of the file. By searching the second local mapping table, the client can directly obtain the communication address of the target Target client for processing the inode forwarding request, and send the forwarding message to the target Target client.

[0061] Through the embodiments of the present application, by introducing the second identification information of the target metadata service and the preset second local mapping table, the client can quickly determine the target centralized control client without additional interaction with the MDS, accelerating the execution speed of the read and write operations. At the same time, the stability and predictability of the concurrent read and write requests are improved through the fixed mapping relationship.

[0062] In some exemplary embodiments, after forwarding the read / write forwarding message to the target centralized control client as described above, the method further includes: when receiving a permission update message broadcast by the target metadata service, deleting the forwarding flag from the latest metadata information to obtain new latest metadata information, and directly reading the file to be read / written on the disk; the permission update message indicates that the file permission of the file to be read / written is switched from a non-exclusive permission to an exclusive permission.

[0063] After the client forwards the read / write forwarding message to the target centralized control client, if the file to be read / written becomes open by a single client again, the target MDS can broadcast a permission update message to the client. When the permission update message indicates that the file permission of the file to be read / written has been switched from a non-exclusive permission to an exclusive permission, the client can immediately delete the forwarding flag from the stored latest metadata information. This deletion operation updates the metadata information in the client cache to reflect the latest state of the file permission. The client can directly access the file to be read / written on the disk, achieving faster read / write operations.

[0064] For example, when two or more clients simultaneously attempt to read / write the same file to be read / written, after the target MDS detects concurrent access, it recovers the cache permissions of all clients and notifies all clients that have opened the file to switch to the non-exclusive permission state. At this time, the client can set a forwarding flag and forward all read / write requests to the target centralized control client for processing. However, when one of the clients obtains the exclusive permission for the file or when the file to be read / written becomes open by a single client again, the target MDS can broadcast a permission update message. The client that receives this message can immediately delete the forwarding flag, update the local metadata cache, and directly read from or write to the file on the disk, without the need to forward the read / write request to the target centralized control client.

[0065] For another example, there are the following two scenarios where the permission of the current client can be updated: Scenario 1: When the current client actively opens a file to be read and written in a read-write manner, it can send an open metadata request to the target MDS. When the target MDS replies to the metadata request, it can carry the permission information granted to the current client, and the permission is updated to the local client cache. Scenario 2: When another client also opens the file to be read and written, the target MDS can actively send a permission revocation or authorization message to the current client, and the permission is updated to the local client cache. In the above two scenarios, the current client compares the change in the exclusive permission before and after the inode information permission change to determine whether there is a situation where multiple clients open the same file at the same time. If the inode exclusive permission (excl) is revoked by the target MDS, it means that multiple clients have opened this file at the same time, and the current client sets a read-write forwarding flag; if the inode exclusive permission is re-granted by the target MDS, it means that only the current client exclusively occupies this file, and the read-write forwarding flag is cancelled. The forwarding flag is saved in the current client inode cache and is only used locally by the current client and is not synchronized to the target MDS.

[0066] Through the embodiments of the present application, by dynamically adjusting the forwarding flag of the local cache, it is possible to immediately cancel the forwarding process when the file permission switches from a non-exclusive state to an exclusive state, allowing the client to directly read and write the file on the disk. This not only improves the efficiency of read and write operations but also reduces the message communication between the client and the target centralized control client, alleviating the system burden. In a high-concurrency scenario where the client permissions change frequently, unnecessary forwarding operations can be effectively avoided.

[0067] In some exemplary embodiments, the read-write forwarding message includes at least one of the following: a read forwarding message and a write forwarding message; the read forwarding message is forwarded to the target centralized control client to perform a read operation on the file to be read and written in the disk through the target centralized control client, and the write forwarding message is forwarded to the target centralized control client to perform a write operation on the file to be read and written in the disk through the target centralized control client; in the case where the forwarding flag of the file to be read and written is stored in the latest metadata information, constructing the read-write forwarding message includes Step 1 and Step 2.

[0068] 1. When the forwarding flag of the file to be read and written is stored in the latest metadata information and the read-write forwarding message represents a read operation on the file to be read and written, fill the first preset field in the first forwarding message to obtain the read forwarding message; the first preset field includes the first identification information of the file to be read and written, the read operation type, the read start offset address, the read data length, and the memory address for storing the read data content.

[0069] Second, store the forwarding flag of the file to be read and written in the latest metadata information. When the read / write forwarding message indicates a write operation on the file to be read and written, fill in the second preset field in the second forwarding message to obtain the write forwarding message. The second preset field includes the first identification information, write operation type, write start offset address, write data length, and the byte stream of the data content to be written.

[0070] When it is detected that the forwarding flag of the file to be read and written is stored in the latest metadata information and a read operation is performed on the file to be read and written, the first preset field can be filled in the first forwarding message to generate the read forwarding message. The first preset field includes the first identification information (e.g., inode number) of the file to be read and written, read operation type, read start offset address, read data length, and the memory address for storing the read data content, so that the target centralized control client can more accurately understand the read request (including the location of the file, the read range, and the return location of the data result), and thus can efficiently execute the read operation and return the read data.

[0071] For example, the client first obtains the forwarding flag in the inode information corresponding to the file to be read and written. If forwarding is required, it constructs a read forwarding message and fills in the first preset field such as the inode of the file, read operation type, read start offset address, read data length, and the memory address allocated for storing the read data content into the first forwarding message to obtain the read forwarding message. The client obtains the target MDS to which the inode of the file to be read and written belongs, finds out the communication address of the corresponding target centralized control client according to the target MDS, and sends the read forwarding message to the target centralized control client. At the same time, a unique and incrementing id is assigned to each read forwarding message and stored in the Initiator client cache, and then deleted from the cache after receiving the message reply.

[0072] When it is detected that the forwarding flag of the file to be read and written is stored in the latest metadata information and a write operation is performed on the file to be read and written, the second preset field can be filled in the second forwarding message to generate the write forwarding message. The second preset field also includes the first identification information (e.g., inode number) of the same file to be read and written, write operation type, write start offset address, write data length, and the byte stream of the data content to be written, so that after receiving the write forwarding message, the target centralized control client can more accurately write the byte stream of the data content to be written to the disk according to the specified offset address and data length, ensuring the consistency and integrity of the written data.

[0073] For another example, the client first obtains the forwarding flag in the inode information corresponding to the file to be read and written. If forwarding is required, a write forwarding message is constructed, and the second preset fields such as the inode of the file, the write operation type, the write start offset address, the write data length, and the data content byte stream to be written are filled into the forwarding message to obtain the write forwarding message. The target MDS to which the inode of the file to be read and written belongs is obtained, and according to the target MDS, the communication address of the corresponding target centralized control client is found, and the write forwarding message is sent to the target centralized control client. At the same time, a unique and incrementing id is assigned to each write forwarding message and stored in the client cache, and then deleted from the cache after receiving the message reply.

[0074] For another example, as Figure 3 shown, in the case where multiple clients read and write the same file to be read and written simultaneously, when client A needs to read the file, after detecting the forwarding flag, a read forwarding message is constructed, and the inode number of the file, the read operation type, the read start address, the read length, and the memory address are filled in, and then sent to the target centralized control client. The Target client (target centralized control client) receives the read and write forwarding messages sent by multiple clients (Initiator clients) and processes the read and write requests serially in order. The Target client parses the following content from the read and write forwarding messages: file inode number, operation type, start offset address, read and write length, written data, etc. The Target client obtains the latest metadata from the MDS according to the inode number in the read and write forwarding message, and then performs read or write data operations according to the offset address and the read and write length. Since all read and write requests can be processed serially, strong consistency of multi-client read and write data can be guaranteed.

[0075] The target centralized control client receives the read forwarding message, reads the data from the disk according to the specified parameters, and returns the read result to the specified memory address of client A. Similarly, when client B needs to write data, a write forwarding message is constructed, including the inode number, the write operation type, the write address, the write length, and the data content byte stream, and sent to the target centralized control client. The target centralized control client serially executes the write operation according to the message content.

[0076] In an alternative embodiment, in the case of performing a read or write operation, it is determined whether there is a forwarding flag in the file to be read and written. If there is no flag, local read and write are performed, that is, the client uses its local cache or buffer to perform the read and write operations instead of directly interacting with the remote storage node (such as OSD).

[0077] Through the embodiments of the present application, read-forward messages and write-forward messages are constructed and transmitted between clients, realizing the effective communication and processing of read and write operation requests, and avoiding data conflicts and inconsistencies caused by concurrent read and write operations of multiple clients. This not only ensures the sequentiality of data operations, but also reduces the computational pressure on the MDS, improving the read and write efficiency and data security of the distributed file system in high-concurrency scenarios, thereby providing a more stable and efficient file access experience.

[0078] In some exemplary embodiments, the read-write forward message is assigned with third identification information; after the above read-write forward message is forwarded to the target centralized control client, the above method further includes steps one to three.

[0079] 1. Save the read-write forward message and the third identification information into the storage structure, and wait for the forward message response message from the target centralized control client.

[0080] 2. In the case that the forward message response message is not received after timeout, resend the read-write forward message.

[0081] 3. In the case that the forward message response message is received within the specified time range, delete the read-write forward message and the third identification information from the storage structure.

[0082] The above third identification information may be a unique identifier assigned to each read-write forward message, used to track the status of the message within the client. This third identification information corresponds one-to-one with the read-write forward message, enabling the client to effectively identify and manage each forward message when waiting for the forward message response message from the target centralized control client.

[0083] In this embodiment, the storage structure may refer to the data structure within the client for saving the forward message and its third identification information. This data structure may be a hash table indexed based on the message ID or a similar data organization form. For example, the storage structure may be a map structure. This storage structure can ensure that after the client sends the read-write forward message, it can quickly locate and manage the life cycle of the forward message according to the third identification information until the forward message response message from the target centralized control client is received. The forward message response message may be the message returned by the target centralized control client to the client after processing the read-write forward message sent by the client, used to confirm the success or failure of the read and write operations, and may also carry feedback information on the read data or write status.

[0084] In this embodiment, after the client sends the read-write forward request, it is in a waiting state. When the forward message response message replied by the target Target client is received, the waiting ends, and according to the read-write result in the forward message response message, it is decided whether to update the metadata. Thus, the normal process of forwarding read and write ends.

[0085] After sending a read / write request to the target centralized control client, if no response is received within a preset time, or if it is detected that the target centralized control client has failed, etc., the client can automatically resend the read / write forwarding message to a new target centralized control client. Before that, the client can save the unique identifier (message ID) of each read / write forwarding message in the local cache and will not delete it until a successful response is received.

[0086] In an optional embodiment, after the forwarding message response message is returned, for a read request, the result of the read operation and the data read can be obtained from the forwarding message response (reply) message; for a write request, the result of the write operation can be obtained from the forwarding message response message. In addition, after the client sends a read / write forwarding request and is in a waiting state, when it receives the forwarding message response message replied by the target centralized control client, it ends the waiting and decides whether to update the metadata according to the read / write result in the forwarding message response message.

[0087] For example, each forwarding message sent by the client will be saved in a map structure and indexed by a globally unique message ID. Only when the forwarding message reply message from the target Target side is received will the forwarding message ID and the corresponding message content be deleted from the map structure. The forwarding message IDs in the map structure are regularly detected. If there are forwarding messages that have not been cleared for more than 5s, it is considered that this forwarding message is abnormal and needs to be resent to ensure data integrity.

[0088] In another optional embodiment, during the multi-client message forwarding process, if problems such as system failures or network failures cause message loss, detection and retransmission are required. To implement this function, a global message cache is maintained on the client. After all forwarding messages are sent, the forwarding message IDs are saved in the cache, and the forwarding message IDs will only be deleted from the cache when the forwarding message reply messages from the target Target client are successfully received. In addition, the forwarding message can adopt a local cache retransmission mechanism. Only when it is confirmed that the forwarding message has been correctly processed by the target centralized control client will the cache of the forwarding message body be released; otherwise, the cache will be used for retransmission.

[0089] Exemplarily, Figure 4 is a timing diagram of an optional read / write method for disk files provided according to an embodiment of the present application, as Figure 4As shown, the client determines the target metadata service according to the file to be read / written, and sends an open metadata request for the file to be read / written to the target metadata service. When the target metadata service detects that multiple clients open the same file to be read / written, the target metadata service sends a metadata response message to the client. The metadata response message includes the original metadata information of the file to be read / written and the file permission information. When the file permission information indicates non-exclusive permission (i.e., the target metadata service reclaims the cache permission of the client and grants direct read / write permission), the client sets a forwarding flag in the original metadata information of the file to be read / written. When the client performs subsequent read / write operations, it determines whether there is a forwarding flag in the original metadata information of the file to be read / written. If there is a forwarding flag, it obtains the target metadata service ID, determines the target centralized control client, forwards the read / write forwarding message to the target centralized control client, starts timing, and stores the read / write forwarding message in the local storage structure of the client. The target centralized control client performs read / write operations on the file to be read / written on the disk, and the disk feeds back the result of the read / write operation to the target centralized control client. The target centralized control client generates a forwarding message response message and sends it to the client. If the client receives a reply within the specified time, it deletes the read / write forwarding message from the storage structure to release the cache. If the client does not receive a reply after the timeout, it resends the read / write forwarding message.

[0090] Through the embodiments of the present application, by allocating the third identification information to the read / write forwarding message and managing it in the local storage structure of the client, the effective tracking of the message status and the automatic retransmission in case of exceptions are realized, significantly enhancing the robustness in the multi-client concurrent scenario. Even in the face of network fluctuations or target client failures, it can ensure that the read / write requests are finally executed, and the integrity and consistency of the data are effectively guaranteed. At the same time, the storage structure avoids the disorderly management of messages and resource waste, enabling the client to manage the message queue more efficiently and reducing unnecessary message processing delays.

[0091] In an optional embodiment, in a network environment, the transmission of data packets may be delayed, especially under the conditions of a wide area network (WAN), which may lead to a temporary inconsistency between the information cached locally by the client and the state of the target centralized control client. In this case, even if the target centralized control client successfully executes the read / write operation, the client may not update the state in time locally, resulting in result differences. Therefore, to solve this problem, in this embodiment, after forwarding the read / write forwarding message to the target centralized control client, the target centralized control client performs read / write operations on the file to be read / written on the disk, and returns the forwarding message response message to the client, and the client verifies the forwarding message response message returned by the target centralized control client.

[0092] For example, the forwarded message response message includes a data checksum or digest information (such as an MD5 hash value). After receiving the forwarded message response message, the client not only checks the result of the read / write operation but also compares the returned data with the data checksum in the returned result. When constructing a read / write forwarding message (such as a read forwarding message), the client can first perform a hash processing on the content to be read / written in the original metadata information of the file to be read / written to obtain a local data checksum, add the local data checksum to the read / write forwarding message, and send the forwarding message containing the local data checksum to the target centralized control client. The target centralized control client receives the forwarding message, performs the read / write operation, and after the operation, performs a hash calculation on the read or written data to generate a non-local data checksum, and encapsulates the non-local data checksum and the read / write data together in the forwarded message response message. After the client receives the forwarded message response message within the specified time, it compares the non-local data checksum in the forwarded message response message with the local data checksum in the local cache to verify the consistency and integrity of the data. If the verification fails, that is, the non-local data checksum in the forwarded message response message does not match the local data checksum preprocessed locally, the client will mark this read / write operation as failed and resend the read / write forwarding request. At the same time, the client can also notify the target centralized control to perform error troubleshooting to ensure the integrity of the data channel.

[0093] Through this embodiment, by including the local data checksum in the read / write forwarding message and comparing it after processing the data, it not only enhances the reliability and security of the data read / write operation in the distributed file system but also optimizes the performance of the entire system, reduces the burden on key components (such as MDS), thereby improving the overall operation efficiency and user experience of the system.

[0094] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0095] According to another aspect of the embodiments of the present application, there is also provided a read / write device for disk files. The read / write device for disk files can be used to implement the read / write method for disk files provided in any of the above embodiments, and those that have been described will not be repeated. Although the devices described in the following embodiments are preferably implemented in software, implementation by hardware, or a combination of software and hardware is also possible and contemplated.

[0096] Figure 5 is a structural block diagram of an optional read / write device for disk files according to the embodiments of the present application, as Figure 5As shown in [figure], the read / write device for disk files is applied to a client, and includes a first execution unit 502, a second execution unit 504, and a third execution unit 506.

[0097] The first execution unit 502 is configured to obtain the latest metadata information of the file to be read / written from the local cache.

[0098] The second execution unit 504 is configured to construct a read / write forwarding message and determine a target centralized control client when a forwarding flag of the file to be read / written is stored in the latest metadata information; the forwarding flag is cached in the latest metadata information when multiple clients simultaneously initiate an open metadata request for the file to be read / written, and the forwarding flag indicates that the read / write request for the file to be read / written is forwarded to the target centralized control client; the target centralized control client refers to the client that uniformly processes the concurrent read / write requests of the multiple clients for the file to be read / written.

[0099] The third execution unit 506 is configured to forward the read / write forwarding message to the target centralized control client, so as to perform read / write operations on the file to be read / written in the disk through the target centralized control client.

[0100] Through the embodiments of the present application, when multiple clients initiate an open metadata request for the same file to be read / written, a forwarding flag of the file to be read / written is stored in the metadata information to obtain the latest metadata information, where the forwarding flag is used to indicate that when the client performs subsequent read / write operations, the read / write request for the file to be read / written is forwarded to the target centralized control client; the forwarding flag is set during permission update, so as to determine whether read / write forwarding is required according to this forwarding flag during read / write operations, without frequent permission granting and revocation message communication with the metadata service, and without frequent lock state switching, reducing the interaction pressure between the metadata service and the client, and performing read / write operations on the file to be read / written through the target centralized control client, eliminating data chaos that may be caused by concurrent operations, so that the read / write requests of all clients can be processed by the target centralized control client, avoiding multiple clients writing to the disk simultaneously, reducing frequent interactions between multiple clients and the metadata service, and reducing the load on the metadata service. Therefore, the problem that the interaction pressure between the metadata service and multiple clients in the related art is large, resulting in slow system response time and small throughput, can be solved.

[0101] It should be noted that the first execution unit 502 in this embodiment can be used to execute the above step S202, the second execution unit 504 in this embodiment can be used to execute the above step S204, and the third execution unit 506 in this embodiment can be used to execute the above step S206.

[0102] In an exemplary embodiment, the apparatus further includes: a fourth execution unit configured to determine a target metadata service corresponding to a file to be read and written according to the file to be read and written; the target metadata service is used to manage the read and write permissions of the file to be read and written; a fifth execution unit configured to send an open metadata request to the target metadata service and receive a metadata response message fed back by the target metadata service; the metadata response message includes the original metadata information of the file to be read and written and file permission information; a sixth execution unit configured to, when the file permission information represents non-exclusive permission, cache a forwarding flag in the original metadata information to obtain the latest metadata information.

[0103] In an exemplary embodiment, the file to be read and written is assigned a first identification information; the fourth execution unit includes: a first execution module configured to find a preset first local mapping table according to the first identification information to obtain the target metadata service; the first local mapping table stores the mapping relationships between different files and metadata services.

[0104] In an exemplary embodiment, the target metadata service is assigned a second identification information; the second execution unit 504 includes: a second execution module configured to find a preset second local mapping table according to the second identification information to determine the target centralized control client; the second local mapping table stores the mapping relationships between different metadata services and different centralized control clients.

[0105] In an exemplary embodiment, the above apparatus further includes: a seventh execution unit configured to, when receiving a permission update message broadcast by the target metadata service, delete the forwarding flag from the latest metadata information to obtain new latest metadata information, and directly read the file to be read and written on the disk; the permission update message represents that the file permission of the file to be read and written is switched from non-exclusive permission to exclusive permission.

[0106] In an exemplary embodiment, the read and write forwarding message includes at least one of the following: a read forwarding message and a write forwarding message; the second execution unit 504 further includes: a third execution module configured to, when the latest metadata information stores a forwarding flag of the file to be read and written and the read and write forwarding message represents a read operation on the file to be read and written, fill a first preset field in the first forwarding message to obtain the read forwarding message; the first preset field includes the first identification information of the file to be read and written, the read operation type, the read start offset address, the read data length, and the memory address storing the read data content; a fourth execution module configured to, when the latest metadata information stores a forwarding flag of the file to be read and written and the read and write forwarding message represents a write operation on the file to be read and written, fill a second preset field in the second forwarding message to obtain the write forwarding message; the second preset field includes the first identification information, the write operation type, the write start offset address, the write data length, and the byte stream of the data to be written.

[0107] In an exemplary embodiment, a read / write forwarding message is assigned with third identification information; the apparatus further includes: a first storage unit, configured to store the read / write forwarding message and the third identification information into a storage structure, and wait for a forwarding message response message from a target centralized control client; a first sending unit, configured to re-send the read / write forwarding message when a forwarding message response message is not received after a timeout; an eighth execution unit, configured to delete the read / write forwarding message and the third identification information from the storage structure when a forwarding message response message is received within a specified time range.

[0108] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor is configured to execute the steps in any of the above method embodiments through the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, where the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0109] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be elaborated herein.

[0110] According to another aspect of the embodiments of the present application, a computer program product is further provided, where the computer program product includes computer programs / instructions, and the computer programs / instructions include program codes for executing the method shown in the flowchart.

[0111] Figure 6 Schematically shows a block diagram of a computer system of an electronic device for implementing the embodiments of the present application. As Figure 6 shown, the computer system 600 includes a central processing unit (CPU) 601, which can execute various appropriate actions and processes according to the program stored in a read-only memory (ROM) 602 or the program loaded from a storage section 608 into a random access memory (RAM) 603. In the random access memory 503, various programs and data required for system operations are also stored. The central processing unit 601, the read-only memory 602, and the random access memory 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0112] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT for short), a liquid crystal display (LCD for short), etc. and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a local area network card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as required. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as required so that a computer program read from it can be installed into the storage section 608 as required.

[0113] Specifically, according to an embodiment of the present application, the processes described in each method flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit 601, various functions provided by the embodiments of the present application are executed. The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0114] It should be noted that Figure 6 the computer system 600 of the electronic device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present application.

[0115] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. Thus, the present application is not limited to any specific combination of hardware and software.

[0116] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various modifications and changes can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application shall be included within the protection scope of the present application.

Claims

1. A method for reading and writing disk files, characterized in that, Applied to the client, including: Obtain the latest metadata information of the file to be read and written from the local cache; When a forwarding flag of the file to be read and written is stored in the latest metadata information, construct a read / write forwarding message and determine the target centralized control client; the forwarding flag is cached in the latest metadata information when multiple clients simultaneously initiate an open metadata request for the file to be read and written, and the forwarding flag indicates that the read / write request for the file to be read and written is forwarded to the target centralized control client; the target centralized control client refers to the client that uniformly processes the concurrent read / write requests of the multiple clients for the file to be read and written; Forward the read / write forwarding message to the target centralized control client to perform read / write operations on the file to be read and written in the disk through the target centralized control client.

2. The method according to claim 1, characterized in that, Before obtaining the latest metadata information of the file to be read and written from the local cache, the method further includes: Determine a target metadata service corresponding to the file to be read and written according to the file to be read and written; the target metadata service is used to manage the read / write permissions of the file to be read and written; Send an open metadata request to the target metadata service and receive a metadata response message fed back by the target metadata service; the metadata response message includes the original metadata information of the file to be read and written and the file permission information; When the file permission information indicates non-exclusive permission, cache the forwarding flag in the original metadata information to obtain the latest metadata information.

3. The method according to claim 2, wherein The file to be read and written is assigned a first identification information; determining a target metadata service corresponding to the file to be read and written according to the file to be read and written includes: Find a preset first local mapping table according to the first identification information to obtain the target metadata service; the first local mapping table stores the mapping relationship between different files and metadata services.

4. The method according to claim 2, wherein The target metadata service is assigned a second identification information; determining the target centralized control client includes: Find a preset second local mapping table according to the second identification information to determine the target centralized control client; the second local mapping table stores the mapping relationship between different metadata services and different centralized control clients.

5. The method according to claim 2, wherein After forwarding the read / write forwarding message to the target centralized control client, the method further includes: When receiving a permission update message broadcast by the target metadata service, delete the forwarding flag from the latest metadata information to obtain the new latest metadata information, and directly read the file to be read and written in the disk; the permission update message indicates that the file permission of the file to be read and written is switched from non-exclusive permission to exclusive permission.

6. The method according to claim 1, wherein The read / write forwarding message includes at least one of the following: a read forwarding message and a write forwarding message; when a forwarding flag of the file to be read and written is stored in the latest metadata information, constructing a read / write forwarding message includes: A forwarding flag of the file to be read and written is stored in the latest metadata information, and when the read / write forwarding message represents a read operation on the file to be read and written, a first preset field is filled in the first forwarding message to obtain the read forwarding message; the first preset field includes first identification information of the file to be read and written, a read operation type, a read start offset address, a read data length, and a memory address for storing the read data content. A forwarding flag of the file to be read and written is stored in the latest metadata information, and when the read / write forwarding message represents a write operation on the file to be read and written, a second preset field is filled in the second forwarding message to obtain the write forwarding message; the second preset field includes the first identification information, a write operation type, a write start offset address, a write data length, and a byte stream of the data content to be written.

7. The method according to claim 1, wherein The read / write forwarding message is assigned a third identification information; after the read / write forwarding message is forwarded to the target centralized control client, the method further includes: Saving the read / write forwarding message and the third identification information to a storage structure, and waiting for a forwarding message response message from the target centralized control client; When the forwarding message response message is not received after a timeout, resending the read / write forwarding message; When the forwarding message response message is received within a specified time range, deleting the read / write forwarding message and the third identification information from the storage structure.

8. A disk file reading and writing device, characterized in that, Applied to a client, including: A first execution unit, configured to obtain the latest metadata information of the file to be read and written from a local cache; A second execution unit, configured to construct a read / write forwarding message and determine a target centralized control client when the forwarding flag of the file to be read and written is stored in the latest metadata information; the forwarding flag is cached to the latest metadata information when multiple clients simultaneously initiate an open metadata request for the file to be read and written, and the forwarding flag represents that the read / write request for the file to be read and written is forwarded to the target centralized control client; the target centralized control client refers to a client that uniformly processes the concurrent read / write requests of the multiple clients for the file to be read and written; A third execution unit, configured to forward the read / write forwarding message to the target centralized control client, so as to perform read and write operations on the file to be read and written in the disk through the target centralized control client.

9. An electronic device, characterized in that, Including: A memory, configured to store a computer program; A processor, configured to implement the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images