Disk file reading and writing method and device, electronic device and storage medium

Abstract

The present application discloses a disk file reading and writing method and device, electronic device and storage medium, and relates to the field of computer technology. The method includes: constructing a read-write forwarding message when a forwarding mark of a file to be read and written is stored in the latest metadata information of the file to be read and written; the forwarding mark is cached in the latest metadata information when multiple clients simultaneously initiate metadata open requests for the file to be read and written, and the forwarding mark indicates that the read and write requests for the file to be read and written are forwarded to a target centralized control client; the target centralized control client refers to a client that uniformly processes concurrent read and write requests from multiple clients for the file to be read and written; forwarding the read and write forwarding message to the target centralized control client, so that the target centralized control client can perform read and write operations on the file to be read and written on the disk. This application solves the problem in the related art that the interaction pressure between the metadata service and multiple clients is high, resulting in slow system response time and low throughput.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for reading and writing disk files, an electronic device, and a storage medium. Background Art

[0002] Ceph is a unified distributed storage system that provides block storage, object storage, and file system storage services. Metadata Services (MDS), the metadata service for the Ceph file system, processes metadata for the file system storage service and coordinates access to the storage cluster. Object Storage Devices (OSDs), Ceph components, are daemons primarily responsible for responding to client requests and returning specific data. A Ceph cluster typically has multiple OSDs, one for each disk.

[0003] In large-scale data centers and cloud computing environments, Ceph, due to its powerful data storage and management capabilities, is widely used in high-concurrency scenarios such as large-scale video streaming, big data analytics, and cloud storage services. Traditionally, Ceph maintains data consistency through MDS permission control and OSD centralized control of write order. However, when other storage backends are used instead of OSDs, the OSD centralized control function is lost, and multiple clients concurrently reading and writing the same file may lead to data inconsistency.

[0004] In related technologies, when multiple clients read and write simultaneously, MDS implements direct reading and writing through permission control. Although this ensures basic data consistency, MDS needs to frequently communicate with all clients to grant and revoke permissions, and needs to frequently switch lock states, which significantly increases the computing pressure of MDS, making it a performance bottleneck in high-concurrency scenarios, affecting the response time and throughput of the entire system. Summary of the Invention

[0005] The present application provides a disk file reading and writing method and device, electronic device and storage medium, to at least solve the problem in the related art that the metadata service interacts with multiple clients under great pressure, resulting in slow system response time and low throughput.

[0006] The present application provides a disk file reading and writing method, which is applied to a client, comprising: obtaining the latest metadata information of a file to be read and written from a local cache; constructing a read and write forwarding message and determining a target centralized control client when a forwarding mark of the file to be read and written is stored in the latest metadata information; the forwarding mark is cached in the latest metadata information when multiple clients simultaneously initiate metadata open requests for the file to be read and written, and the forwarding mark indicates that the read and write requests for the file to be read and written are forwarded to the target centralized control client; the target centralized control client refers to a client that uniformly processes concurrent read and write requests of the multiple clients for the file to be read and written; and forwarding the read and write forwarding message to the target centralized control client, so that the read and write operations on the file to be read and written on the disk are performed through the target centralized control client.

[0007] The present application also provides a disk file reading and writing device, which is applied to a client, including: a first execution unit, used to obtain the latest metadata information of the file to be read and written from a local cache; a second execution unit, used to construct a read and write forwarding message and determine a target centralized control client when a forwarding mark of the file to be read and written is stored in the latest metadata information; the forwarding mark is cached in the latest metadata information when multiple clients simultaneously initiate metadata open requests for the file to be read and written, and the forwarding mark indicates that the read and write requests for the file to be read and written are forwarded to the target centralized control client; the target centralized control client refers to a client that uniformly processes concurrent read and write requests of the multiple clients for the file to be read and written; a third execution unit, used to forward the read and write forwarding message to the target centralized control client, so that the read and write operations on the file to be read and written on the disk are performed through the target centralized control client.

[0008] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing any of the above methods when executing the computer program.

[0009] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any of the above methods is implemented.

[0010] Through this application, when multiple clients initiate metadata open requests for the same file to be read or written, a forwarding tag for the file to be read or written is stored in the metadata information to obtain the latest metadata information. The forwarding tag is used to instruct the client to forward the read or write request for the file to be read or written to the target centralized control client when performing subsequent read or write operations. The forwarding tag is set when permissions are updated so that when performing read or write operations, the forwarding tag is used to determine whether read or write forwarding is required. This eliminates the need for frequent permission granting and revoking message communications with the metadata service and frequent lock state switching, reducing the interaction pressure between the metadata service and the client. Furthermore, by having the target centralized control client perform read and write operations on the file to be read or written, data confusion caused by concurrent operations is eliminated, so that all client read and write requests can be processed by the target centralized control client, avoiding multiple clients writing to the disk at the same time, reducing frequent interactions between multiple clients and the metadata service, alleviating the load on the metadata service, and improving the system's response time and throughput. Therefore, the problem of high interaction pressure between the metadata service and multiple clients in the related art, resulting in slow system response time and low throughput, can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 This is a schematic diagram of an application scenario of a disk file reading and writing method provided according to an embodiment of the present application.

[0013] Figure 2 It is a flowchart of a method for reading and writing disk files provided according to an embodiment of the present application.

[0014] Figure 3 This is a flow chart of an optional disk file reading and writing method provided according to an embodiment of the present application.

[0015] Figure 4 This is a timing diagram of an optional disk file reading and writing method provided according to an embodiment of the present application.

[0016] Figure 5 This is a structural block diagram of an optional disk file reading and writing device according to an embodiment of the present application.

[0017] Figure 6 This is a block diagram of a computer system structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0020] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0021] According to one aspect of the embodiment of the present application, a method for reading and writing a disk file is provided. Optionally, in this embodiment, the method for reading and writing a disk file can be applied to, but is not limited to, Figure 1 The system shown includes multiple clients 102, a metadata server 104, and a file storage server 106. The multiple clients 102 communicate with the metadata server 104 and the file storage server 106, respectively, and can be used to provide services (e.g., application services, etc.) for the multiple clients 102 installed on the terminal devices. The metadata server 104 is responsible for managing and coordinating the metadata of the file system, including information such as file location, permissions, and attributes. The file storage server 106 is responsible for storing the actual file data and, through collaboration with the metadata server, efficiently responds to read and write requests to meet large-scale data access needs. A database can be set up on the metadata server 104 and the file storage server 106, or independently of the metadata server 104 and the file storage server 106, to provide data storage services for the metadata server 104 and the file storage server 106.

[0022] The aforementioned network may include, but is not limited to, at least one of the following: a wired network or a wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, or a local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: wireless fidelity (Wi-Fi) and Bluetooth. Client 102 may be, but is not limited to, a personal computer (PC), a mobile phone, a tablet computer, etc. Metadata server 104 and file storage server 106 may be, but are not limited to, cloud servers, server clusters, or other server types.

[0023] The disk file reading and writing method of the embodiment of the present application can be executed by the client 102. Taking the disk file reading and writing method of the embodiment of the present application as an example, Figure 2 FIG. 1 is a flow chart of an optional method for reading and writing a disk file according to an embodiment of the present application, such as Figure 2 As shown, the process of the method may include steps S202 to S206.

[0024] Step S202: Acquire the latest metadata information of the file to be read or written from the local cache.

[0025] Step S204: When a forwarding mark of the file to be read or written is stored in the latest metadata information, a read / write forwarding message is constructed and a target centralized control client is determined; the forwarding mark is cached in the latest metadata information when multiple clients simultaneously initiate metadata open requests for the file to be read or written, and the forwarding mark indicates that the read / write requests for the file to be read or written are forwarded to the target centralized control client; the target centralized control client refers to a client that uniformly processes concurrent read / write requests from multiple clients for the file to be read or written.

[0026] Step S206: forwarding the read / write forwarding message to the target centralized control client, so that the target centralized control client performs read / write operations on the file to be read / written in the disk.

[0027] The disk file reading and writing method in the embodiment of the present application can be applied to the field of distributed storage system technology and to high-concurrency scenarios (for example, large-scale video streaming, big data analysis, and cloud storage services). Ceph is a unified distributed storage system that provides block storage services, object storage services, and file system storage services. Metadata Services (MDS) serves as the metadata service of the Ceph file system and is used to process the metadata of the file system storage service. It also coordinates access to the storage cluster. The client is the interface that interacts with the user in the Ceph system and is responsible for accessing and operating data in the Ceph storage cluster, implementing data access, request forwarding, and interactive interfaces. Object Storage Device (OSD) is a component of Ceph and is a daemon process that is mainly responsible for responding to client requests and returning specific data. Generally, a Ceph cluster will have multiple OSDs, and each disk will correspond to an OSD.

[0028] In deployment scenarios where other storage backends are used instead of OSD, for situations where multiple clients read and write at the same time, the relevant technology is to implement client read and write sequence control through MDS permission control. When MDS determines that multiple clients open a file at the same time, it grants exclusive read and write permissions to each client in the order in which each client applies for write permissions, and other clients wait serially.

[0029] However, in the above-mentioned related technologies, when multiple clients read and write simultaneously, MDS implements direct reading and writing through permission control. Although basic data consistency is guaranteed, MDS needs to frequently communicate with all clients on permission granting and revoking messages, and needs to frequently switch lock states, which significantly increases the computing pressure of MDS, making it a performance bottleneck in high-concurrency scenarios, affecting the response time and throughput of the entire system.

[0030] In order to at least partially solve the above technical problems, in this embodiment, when multiple clients initiate metadata opening requests for the same file to be read or written, a forwarding tag of the file to be read or written is stored in the metadata information, a read / write forwarding message is constructed and the target centralized control client is determined, and a forwarding tag is set when the permissions are updated so that when performing read / write operations, it is determined whether read / write forwarding is required based on this forwarding tag, and the read / write forwarding message is sent to the target centralized control client. After receiving the read / write forwarding message, the target centralized control client processes the read / write operations, eliminating data confusion that may be caused by concurrent operations, so that all client read / write requests can be processed by the target centralized control client, avoiding multiple clients writing to the disk at the same time, reducing the frequent interactions between multiple clients and the metadata service, and reducing the load on the metadata service. Therefore, it can solve the problem in the related art that the MDS needs to frequently communicate permissions with all clients for granting and revoking messages, and needs to frequently switch lock states, affecting the response time and throughput of the entire system.

[0031] When the client actively opens the file to be read or written in read-write mode, it will send an open metadata request to the MDS. The open data request is used to initialize access to the file to be read or written. After the open metadata request is successful, the client can start the subsequent actual read and write operations. When the MDS receives the open data request, it will check the access status of the file to be read or written. In the case that multiple clients simultaneously initiate open metadata requests for the file to be read or written, the MDS will inform the client of the access status of the file to be read or written and the metadata information of the file to be read or written. In the case that the access status of the file to be read or written indicates that multiple clients simultaneously initiate open metadata requests for the file to be read or written, the client sets a forwarding mark in the metadata information of the file to be read or written. The above forwarding mark can be a specific identifier used to indicate that a file is in a state of concurrent access by multiple clients, and indicates that the read and write requests for the file to be read or written are forwarded to the target centralized control client.

[0032] The metadata information described above can be considered a collection of attributes for the file to be read or written, including the file's inode (an inode is a data structure in the file system used to store metadata about files or directories, including information such as the file's size, type, permissions, owner, and modification time), permission status, and other meta-information, stored in the client's local cache. The latest metadata information ensures that the client can obtain the latest status of the file when performing read and write operations. When the latest metadata information contains a forwarding tag for the file to be read or written, all read and write operations for the file will be processed by the target centralized control client, rather than by each client independently.

[0033] The client (the initiating client) constructs a read / write forwarding message after detecting the forwarding flag. This message is used to perform read / write operations on the file to be read / written. The read / write forwarding message may include, but is not limited to, the file's inode information, the operation type (read or write), the offset, the operation length, and the data to be read or written.

[0034] A target centralized control client can refer to a client in a distributed file system architecture that handles concurrent read and write requests from multiple clients for the same file. A target centralized control server can receive read and write forwarding messages from other clients (e.g., initiator clients) and perform read and write operations on files on the underlying disk or storage device according to the instructions and sequence in the read and write forwarding messages. The target centralized control server can be a new client in addition to the multiple clients in the distributed file system architecture.

[0035] The above-mentioned method of determining the target centralized control client can be to use a hash algorithm, take the identification information of the file to be read or written (such as the file name, inode number, etc.) as input, generate a fixed hash value, and then select a centralized control client from a set of preset centralized control clients as the target centralized control client based on this hash value. The method of determining the target centralized control client can also be to use a polling mechanism to select the target centralized control client, that is, each time a read or write forwarding request is received, the next node in the centralized control client list is selected in a certain order in a loop. The method of determining the target centralized control client can also be determined by inode information and MDS. The client calculates the MDS (target metadata service) to which the file to be read or written belongs through the inode information, and then obtains the communication address of the target centralized control client bound to the belonging MDS.

[0036] The read and write forwarding messages are sent to the target centralized control client. After receiving all the forwarding messages, the target centralized control client will perform read or write operations on the files to be read and written on the disk in the order in which the messages arrive. It should be noted that the way in which the target centralized control client performs read and write operations on the files to be read and written on the disk can be that after the target centralized control client receives the read and write requests forwarded from different clients, it can perform serial processing in the order of the requests, or the target centralized control client directly interacts with the disk to perform read and write operations on the files to be read and written. In an embodiment of the present application, the target centralized control client only processes the data read and write disk, and does not update the metadata. After the read and write are completed, it replies to the client with a response message. For a read forwarding request, it can reply with the result of the read data and the content of the read data; for a write forwarding request, it can reply with the length of the written data.

[0037] For example, a target centralized control client is bound to an MDS. Each MDS is bound to a target centralized control client (a Target client). Forwarding messages for files belonging to this MDS are fixedly sent to the Target client for processing. This Target centralized control client, acting as the destination for read and write forwarding requests, is dedicated to processing read and write requests from other clients (initiator clients). It serially completes data read and write operations in the order requested, and responds to the initiator client with the read and write results and the read data via forwarding messages. It is understood that when multiple clients attempt to read and write the same file simultaneously, the MDS can reclaim cache permissions from all clients and grant direct read and write permissions to all clients. After detecting the forwarding flag, the Initiator client forwards the read and write requests to the Target client bound to the MDS that owns the file's inode. The Target client is responsible for serially processing all read and write requests to ensure data consistency.

[0038] Optionally, a client opens a file in read-write mode, obtaining the latest metadata for the file from its local cache. If the file is currently being accessed concurrently by multiple clients, a forwarding flag may be found in its metadata. Upon discovering the forwarding flag, the client constructs a read-write forwarding message containing the file's inode information, the request type (e.g., read or write), the starting address offset, the operation length, and the relevant data content. The client then uses the inode information to determine the identifier and communication address of the target centralized control client and sends the read-write forwarding message to the target centralized control client. The target centralized control client receives read and write requests for files to be read or written from multiple clients and processes them in the order they were received. When the target centralized control client completes the read or write operation on the disk for the file to be read or written, it converts the result and the read data (if it was a read operation) into a reply message and sends it back to the client that issued the read or write request, allowing the client to update its local metadata cache and complete the read or write operation.

[0039] For example, Figure 3 This is a flow chart of an optional disk file reading and writing method provided in an embodiment of the present application, such as Figure 3 As shown, there are two clients, client A and client B. Client A sends a write forwarding message A1 to the target centralized control client, and client B sends a write forwarding message B1 to the target centralized control client. The target centralized control client writes data A2 and data B2 to file 1 on the disk in sequence. After completion, the target centralized control client sends a forwarding message reply A3 to client A and a forwarding message reply B3 to client B.

[0040] According to an embodiment of the present application, when multiple clients initiate metadata open requests for the same file to be read or written, a forwarding tag of the file to be read or written is stored in the metadata information to obtain the latest metadata information, wherein the forwarding tag is used to instruct the client to forward the read or write request for the file to be read or written to the target centralized control client when performing subsequent read or write operations; the forwarding tag is set when the permission is updated so that when performing read or write operations, it is determined whether read or write forwarding is required based on this forwarding tag. This eliminates the need for frequent permission granting and revoking message communications with the metadata service and the need for frequent lock state switching, thereby reducing the interaction pressure between the metadata service and the client. Furthermore, by having the target centralized control client perform read and write operations on the file to be read or written, data confusion caused by concurrent operations is eliminated, so that all client read and write requests can be processed by the target centralized control client, avoiding multiple clients writing to the disk at the same time, reducing the frequent interactions between multiple clients and the metadata service, and alleviating the load on the metadata service. Therefore, the problem of high interaction pressure between the metadata service and multiple clients in the related art, resulting in slow system response time and low throughput, can be solved.

[0041] In some exemplary embodiments, before obtaining the latest metadata information of the file to be read or written from the local cache, the method further includes steps 1 to 3.

[0042] 1. According to the file to be read or written, a target metadata service corresponding to the file to be read or written is determined; the target metadata service is used to manage the read and write permissions of the file to be read or written.

[0043] 2. Send an open metadata request to the target metadata service, and receive a metadata response message fed back by the target metadata service; the metadata response message includes the original metadata information of the file to be read or written and file permission information.

[0044] 3. When the file permission information indicates non-exclusive permission, the forwarding tag is cached in the original metadata information to obtain the latest metadata information.

[0045] Determining the target metadata service corresponding to the file to be read or written means that the client identifies the metadata service (MDS) responsible for managing the read and write permissions for the file to be read or written. Because metadata information for each file in the Ceph distributed file system may be managed by a different MDS, the client can determine the target MDS that manages the read and write permissions for the file to be read or written by calculating or querying the mapping between the file's inode information and the MDS.

[0046] Before a client needs to perform a read / write operation on a file to be read / written, it can first send a metadata open request to the target MDS. The metadata open request includes the identification information of the file to be read / written. After receiving the request, the target MDS can check the current status of the file to be read / written, generate a metadata response message, and return it to the client. The metadata response message can be the response that the metadata service returns to the client after processing the metadata open request sent by the client to the metadata service. For example, the metadata response message can include raw metadata information and file permission information for the file to be read / written. Metadata information can refer to data describing the properties and structure of the file to be read / written. For example, raw metadata information can include, but is not limited to, the identification information of the file to be read / written, the file attributes (e.g., file size, creation time, modification time, access permissions, etc.), file structure information (e.g., the file's block layout, data block storage location, and the relationship between files and directories, etc.), and file permission information (access control information related to the file, such as read, write, and execute permissions). File permission information refers to data that controls the client's access level to a file, typically including read, write, and execute permissions, as well as whether a specific file (the file to be read or written) is set to exclusive access. File permission information is used to determine whether the client is an exclusive client, that is, whether the client can directly read and write the file to be read or written, or whether the read and write forwarding information needs to be forwarded to the target centralized control client for serialization processing. For example, file permission information is controlled and allocated by the target MDS. After the client receives the file permission information via a metadata response message, it can determine whether the client itself can directly operate on the file to be read or written, or whether it needs to forward the read and write forwarding request to the target centralized control client.

[0047] In the case where the file permission information represents exclusive permission, it means that the file to be read and written is only opened by one client in read and write mode, and this client has full operation permission on the file to be read and written, such as reading cache / writing buffer / reading and writing data / modifying metadata, etc. In the case where the file permission information represents non-exclusive permission, it means that the current file to be read and written may be accessed by multiple clients at the same time, and multiple clients only have direct read and direct write permissions. In this case, the client caches the forwarding mark in the cached original metadata information of the file to be read and written to obtain the latest metadata information. The forwarding mark can indicate that all read and write requests for the same file are forwarded to the target centralized control client for processing.

[0048] For example, if a file to be read or written is opened by only one client in read-write mode, that client has full operational permissions for the file (reading cache / writing buffer / reading and writing data / modifying metadata, etc.). This client is considered an exclusive client. In this state, all read and write requests from the exclusive client are processed serially. However, when a file to be read or written is opened by multiple clients simultaneously in read-write mode, the target MDS can reclaim cache permissions from all clients and grant direct read and write permissions. In this case, there is no exclusive client. At this time, each client sets a forwarding flag in the file inode information in the metadata cache. All subsequent read and write requests from the client to read or write the file will trigger the read and write forwarding logic.

[0049] In the case where the file permission message indicates exclusive access, the client can directly obtain or update the file's metadata information from the local cache and directly perform read and write operations on the file on disk. For example, when the client receives a metadata response message in which the file permission information indicates that the client has exclusive access to read and write the file, the client can perform the following operations: if a forwarding mark was previously set for the file to be read or written, the forwarding mark will be cleared at this time; the client updates the original metadata information in the local cache based on the received metadata response message; after confirming the permission status, the client can directly perform read and write operations without constructing a forwarding message or determining the target centralized control client, reducing delays and communication overhead on the read and write paths.

[0050] Through the embodiments of the present application, when a client detects non-exclusive permissions, that is, when a file to be read or written is accessed simultaneously by multiple clients, the client will add a forwarding mark in the locally cached metadata to indicate that the client needs to forward its read or write request to a specific target client rather than directly operate on the data. This avoids data overwrite and inconsistency issues that may be caused by direct concurrent writes, while reducing the lock switching and permission management pressure of the MDS in high-concurrency scenarios. In addition, by saving the forwarding mark in the client's local metadata cache instead of synchronizing with the MDS, the number of interactions between the client and the MDS is reduced, thereby significantly reducing communication overhead.

[0051] In some exemplary embodiments, the file to be read or written is assigned with first identification information; the above-mentioned determination of the target metadata service corresponding to the file to be read or written based on the file to be read or written includes: searching a preset first local mapping table according to the first identification information to obtain the target metadata service; the first local mapping table stores the mapping relationship between different files and metadata services.

[0052] In embodiments of the present application, the first identification information may refer to information used to uniquely identify and locate a file or data object to be read or written. For example, the first identification information may refer to the inode number of the file to be read or written. The inode number may be an internal number used to uniquely identify each file in the Ceph file system and is independent of the file name. Even if the file name changes, the inode number remains unchanged.

[0053] The first local mapping table can be a data structure pre-set locally on the client, used to store the mapping relationship between different files and metadata services. In the Ceph distributed file system, in order to balance the load and improve efficiency, different files can be managed by different MDSs. The first local mapping table records this mapping relationship. By querying the first local mapping table, the client can quickly determine the target metadata service corresponding to the file to be read or written. Before the client needs to perform a read or write operation on a file to be read or written, it obtains the first identification information of the file to be read or written, and then searches the first local mapping table on the client to determine the target metadata service that matches the file to be read or written.

[0054] For example, when reading or writing a file to be read or written, the local cache can be called to obtain the first identification information of the file to be read or written, such as the inode number. Subsequently, the client uses the inode number as a key value to query the preset first local mapping table. The first local mapping table is filled in when the client is initialized, and records the correspondence between all files and the MDS responsible for their metadata management. The query to the first local mapping table may return the ID of the target MDS associated with the file to be read or written. After obtaining the ID of the target MDS, the client will clearly identify the target MDS that manages the read and write permissions of the file, and can then send a metadata open request to the target MDS.

[0055] Through the embodiment of the present application, the first identification information and the preset first local mapping table are introduced, so that the client can quickly and accurately determine the target metadata service that manages the read and write permissions of the file to be read and written before performing the file opening operation, thereby reducing the frequency and delay of message communication, and improving the system's response speed and concurrent processing capabilities, ensuring that metadata permissions management can be performed efficiently and accurately when multiple clients concurrently read and write the same file.

[0056] In some exemplary embodiments, the target metadata service is assigned second identification information; the above-mentioned determination of the target centralized control client includes: searching a preset second local mapping table according to the second identification information to determine the target centralized control client; the second local mapping table stores the mapping relationship between different metadata services and different centralized control clients.

[0057] The second identification information may refer to the identification information of the target metadata service. For example, the second identification information may be the ID of the target metadata service or any symbol that uniquely identifies the target metadata service. The second identification information may be used to locate the target centralized control client associated with the target metadata service in the client's local mapping table. The identification information of the target metadata service may be set during the initialization or configuration phase.

[0058] The second local mapping table can be a data structure pre-configured on the client to store mappings between different metadata services and different centralized control clients. This second local mapping table ensures that when multiple clients simultaneously initiate metadata open requests for a file to be read or written, read or write operation requests for the file to be read or written are forwarded to the correct target centralized control client for processing. By querying the second local mapping table, the client can quickly determine the target centralized control client bound to the target MDS to which the file to be read or written belongs.

[0059] In this embodiment, after confirming that the read / write message for the file to be read / written needs to be forwarded, the client can use the second identification information of the target metadata service to search the second local mapping table. The search process can utilize data structures such as indexes or hash tables, or Ceph's Controlled, Scalable, Decentralized Placement Strategy (CRUSH) algorithm or other similar mechanisms to ensure efficient and accurate search, thereby determining the target centralized control client bound to the target metadata service.

[0060] For example, after assembling a forward message, the client retrieves the ID of the target metadata service based on the inode number and the address of the target client. After sending the message, the client waits for a response. The client maintains a secondary mapping table that records the address of the target client associated with each MDS ID. Based on the file's inode information, the client calculates the ID of the target MDS (target metadata service) to which the file belongs. By searching the secondary local mapping table, the client directly obtains the communication address of the target client that will handle the inode forwarding request and sends the forwarding message to the target client.

[0061] Through the embodiments of the present application, the second identification information of the target metadata service and the preset second local mapping table are introduced, which enables the client to quickly determine the target centralized control client without additional interaction with the MDS, thereby accelerating the execution speed of read and write operations, and at the same time improving the stability and predictability of concurrent read and write requests through fixed mapping relationships.

[0062] In some exemplary embodiments, after forwarding the read-write forwarding message to the target centralized control client, the method further includes: upon receiving a permission update message broadcast by the target metadata service, deleting the forwarding mark from the latest metadata information, obtaining new latest metadata information, and directly reading the file to be read and written on the disk; the permission update message indicates that the file permission of the file to be read and written is switched from non-exclusive permission to exclusive permission.

[0063] After the client forwards the read / write forwarding message to the target centralized control client, if the file to be read / written becomes open to a single client again, the target MDS can broadcast a permission update message to the client. The permission update message indicates that the file permissions of the file to be read / written have been switched from non-exclusive permissions to exclusive permissions. The client can immediately delete the forwarding mark from the latest stored metadata information. This deletion operation updates the metadata information in the client cache to reflect the latest status of the file permissions. The client can directly access the file to be read / written on disk, achieving faster read and write operations.

[0064] For example, when two or more clients try to read and write the same file to be read or written at the same time, the target MDS detects concurrent access, reclaims the cache permissions of all clients, and notifies all clients that have opened the file to switch to a non-exclusive permission state. At this time, the client can set a forwarding mark to forward all read and write requests to the target centralized control client for processing. However, when one of the clients obtains exclusive permission to the file or when the file to be read or written becomes open by a single client again, the target MDS can broadcast a permission update message. The client that receives this message can immediately delete the forwarding mark, update the local metadata cache, and directly read or write the file on the disk without forwarding the read and write request to the target centralized control client.

[0065] For example, a client can experience permissions updates in the following two scenarios: Scenario 1: When the client actively opens a file for read / write, it can send an open metadata request to the target MDS. The target MDS responds to the metadata request with the permissions granted to the client, updating the permissions in the local client cache. Scenario 2: When other clients also open the file for read / write, the target MDS can proactively send a permission revocation or authorization message to the client, updating the permissions in the local client cache. In both scenarios, the client compares the changes in exclusive permissions before and after the inode information permission change to determine whether multiple clients are simultaneously opening the same file. If the inode exclusive permission (excl) is revoked by the target MDS, it indicates that multiple clients have opened the file simultaneously, and the client sets the read / write forwarding flag. If the inode exclusive permission is regranted by the target MDS, it indicates that the file is exclusively open to the current client, and the read / write forwarding flag is removed. The forwarding flag is stored in the client's inode cache for local use only and is not synchronized with the target MDS.

[0066] By dynamically adjusting the locally cached forwarding flags, the embodiment of the present application can immediately cancel the forwarding process when file permissions switch from non-exclusive to exclusive, allowing the client to directly read and write files on disk. This not only improves the efficiency of read and write operations, but also reduces message traffic between the client and the target centralized control client, alleviating system burden. This can effectively avoid unnecessary forwarding operations in high-concurrency scenarios where client permissions frequently change.

[0067] In some exemplary embodiments, the read-write forwarding message includes at least one of the following: a read forwarding message and a write forwarding message; the read forwarding message is forwarded to the target centralized control client so that the target centralized control client can perform a read operation on the file to be read and written on the disk, and the write forwarding message is forwarded to the target centralized control client so that the target centralized control client can perform a write operation on the file to be read and written on the disk; in the above case where the forwarding mark of the file to be read and written is stored in the latest metadata information, constructing the read-write forwarding message includes step one and step two.

[0068] 1. When a forwarding mark of a file to be read or written is stored in the latest metadata information, and a read / write forwarding message indicates a read operation on the file to be read or written, a first preset field is filled in a first forwarding message to obtain a read forwarding message; the first preset field includes first identification information of the file to be read or written, a read operation type, a read starting offset address, a read data length, and a memory address for storing the read data content.

[0069] 2. When a forwarding mark of a file to be read or written is stored in the latest metadata information and a read / write forwarding message indicates a write operation to be performed on the file to be read or written, a second preset field is filled in a second forwarding message to obtain a write forwarding message; the second preset field includes the first identification information, the write operation type, the write start offset address, the write data length, and the byte stream of the data content to be written.

[0070] When a forwarding tag for a file to be read or written is detected in the latest metadata and a read operation is performed on the file to be read or written, a first preset field can be filled in a first forwarding message to generate a read forwarding message. The first preset field includes first identification information of the file to be read or written (for example, the inode number), the read operation type, the read starting offset address, the read data length, and the memory address for storing the read data content. This allows the target centralized control client to more accurately understand the read request (including the file location, read range, and the return location of the data result), thereby efficiently executing the read operation and returning the read data.

[0071] For example, the client first obtains the forwarding flag from the inode information corresponding to the file to be read or written. If forwarding is required, it constructs a read forwarding message and fills the first preset fields, such as the file's inode, read operation type, read starting offset address, read data length, and the memory address allocated to store the read data, into the first forwarding message to obtain a read forwarding message. The client then obtains the target MDS to which the inode of the file to be read or written belongs. Based on the target MDS, it finds the communication address of the corresponding target centralized control client and sends a read forwarding message to the target centralized control client. Each read forwarding message is assigned a unique and incrementing ID and stored in the Initiator client cache. Upon receiving a reply to the message, the message is deleted from the cache.

[0072] When a forwarding tag for a file to be read or written is detected in the latest metadata information and a write operation is performed on the file to be read or written, a second preset field can be filled in a second forwarding message to generate a write forwarding message. The second preset field also includes the first identification information of the same file to be read or written (e.g., inode number), the write operation type, the write starting offset address, the write data length, and the byte stream of the data content to be written. This allows the target centralized control client to more accurately write the byte stream of the data content to be written to the disk according to the specified offset address and data length after receiving the write forwarding message, thereby ensuring the consistency and integrity of the written data.

[0073] For another example, the client first obtains the forwarding flag from the inode information corresponding to the file to be read or written. If forwarding is required, a write forwarding message is constructed, and the second preset field, such as the file's inode, write operation type, write starting offset address, write data length, and the byte stream of the data content to be written, is filled in the forwarding message to obtain the write forwarding message. The target MDS to which the inode of the file to be read or written belongs is obtained. Based on the target MDS, the communication address of the corresponding target centralized control client is found, and a write forwarding message is sent to the target centralized control client. A unique and incrementing ID is assigned to each write forwarding message and stored in the client cache. After receiving a message reply, the message is deleted from the cache.

[0074] For example, Figure 3 As shown in the figure, when multiple clients simultaneously read and write the same file, client A detects the forwarding flag when it needs to read the file. It constructs a read forwarding message, enters the file's inode number, read operation type, read starting address, read length, and memory address, and then sends it to the target centralized control client. The target client (target centralized control client) receives the read and write forwarding messages from multiple clients (initiator clients) and processes the read and write requests sequentially. The target client parses the read and write forwarding messages to include the file inode number, operation type, starting offset address, read and write length, and written data. Based on the inode number in the read and write forwarding message, the target client retrieves the latest metadata from the MDS and then performs the read or write operation based on the offset address and read and write length. Because all read and write requests can be processed serially, strong consistency of multi-client read and write data is guaranteed.

[0075] The target centralized control client receives the read forwarding message, reads the data from the disk according to the specified parameters, and returns the read result to the specified memory address of client A. Similarly, when client B needs to write data, it constructs a write forwarding message containing the inode number, write operation type, write address, write length, and data content byte stream, and sends it to the target centralized control client. The target centralized control client then serially executes the write operation according to the message content.

[0076] In an optional embodiment, when performing a read or write operation, it is determined whether there is a forwarding mark in the file to be read or written. If there is no mark, local reading and writing are performed, that is, the client uses its local cache or buffer to perform read and write operations instead of directly interacting with a remote storage node (such as OSD).

[0077] Through the embodiments of this application, read forwarding messages and write forwarding messages are constructed and transmitted between clients, enabling the efficient communication and processing of read and write operation requests, avoiding data conflicts and inconsistencies caused by concurrent read and write operations by multiple clients. This not only ensures the sequential nature of data operations, but also reduces the computing pressure on the MDS, improving the read and write efficiency and data security of the distributed file system in high-concurrency scenarios, thereby providing a more stable and efficient file access experience.

[0078] In some exemplary embodiments, the read-write forwarding message is assigned with third identification information; after forwarding the read-write forwarding message to the target centralized control client, the method further includes steps one to three.

[0079] 1. Save the read-write forwarding message and the third identification information in the storage structure, and wait for the forwarding message response message from the target centralized control client.

[0080] 2. If no forwarding message response message is received within a timeout period, the read-write forwarding message is resent.

[0081] 3. When a forwarding message response message is received within a specified time range, the read-write forwarding message and the third identification information are deleted from the storage structure.

[0082] The third identification information can be a unique identifier assigned to each read / write forwarded message, used to track the message status within the client. This third identification information corresponds one-to-one with each read / write forwarded message, allowing the client to effectively identify and manage each forwarded message while awaiting a forwarded message response message from the target centralized control client.

[0083] In this embodiment, the storage structure may refer to a data structure within the client for storing forwarded messages and their third identification information. The data structure may be a hash table or similar data organization form based on a message ID index. For example, the storage structure may be a map structure. This storage structure ensures that after a read-write forwarded message is sent, the client can quickly locate and manage the lifecycle of the forwarded message based on the third identification information until a forwarded message response message is received from the target centralized control client. The forwarded message response message may be a message returned to the client by the target centralized control client after processing the read-write forwarded message sent by the client to confirm the success of the read-write operation, and may also carry feedback information on the read data or write status.

[0084] In this embodiment, after sending a read / write forwarding request, the client enters a waiting state. When the target client receives a forwarding message response message, the waiting state ends. Based on the read / write result in the forwarding message response message, the client decides whether to update the metadata. At this point, the normal forwarding read / write process ends.

[0085] After sending a read / write request to a target centralized control client, if no response is received within a preset time, or if the target centralized control client is detected to be faulty, the client can automatically resend the read / write forwarding message to a new target centralized control client. Prior to this, the client can store the unique identifier (message ID) of each read / write forwarding message in a local cache and delete it only after receiving a successful response.

[0086] In one optional embodiment, after the forwarded message response message returns, the read operation result and read data can be obtained from the forwarded message reply message for read requests; the write operation result can be obtained from the forwarded message reply message for write requests. Furthermore, after the client sends a read or write forwarded request, it enters a waiting state. Upon receiving the forwarded message reply message from the target centralized control client, the waiting state ends and the decision on whether to update the metadata is made based on the read or write results in the forwarded message reply message.

[0087] For example, after each forwarded message is sent by the client, it will be saved in a map structure and indexed by the globally unique message id. Only when a reply message to the forwarded message is received from the target end will the forwarded message id and the corresponding message content be deleted from the map structure. The forwarded message id in the map structure is checked regularly. If there is a forwarded message that has not been cleared for more than 5 seconds, it is considered that the forwarded message is abnormal and needs to be resent to ensure data integrity.

[0088] In another optional embodiment, during the multi-client message forwarding process, if a system failure or network failure causes message loss, detection and retransmission are required. To implement this function, a global message cache is maintained on the client. After all forwarded messages are sent, the forwarded message ID is saved in the cache. The forwarded message ID is only deleted from the cache after a forwarded message reply message is successfully received from the target client. In addition, a local cache retransmission mechanism can be used for forwarded messages. The forwarded message body cache is released only when it is confirmed that the forwarded message has been correctly processed by the target centralized control client. Otherwise, the cache is used for retransmission.

[0089] For example, Figure 4 FIG. 1 is a timing diagram of an optional disk file reading and writing method provided in an embodiment of the present application, such as Figure 4As shown, a client determines a target metadata service based on the file to be read or written and sends an open metadata request for the file to the target metadata service. When the target metadata service detects that multiple clients have opened the same file, it sends a metadata response message to the client. This metadata response message includes the original metadata information of the file to be read or written and the file permissions. If the file permissions indicate non-exclusive permissions (i.e., the target metadata service reclaims the client's cached permissions and grants direct read and write permissions), the client sets a forwarding flag in the original metadata information of the file to be read or written. When performing subsequent read or write operations, the client determines whether the forwarding flag exists in the original metadata information of the file to be read or written. If the forwarding flag exists, the client obtains the target metadata service ID, identifies the target centralized control client, forwards the read / write forwarding message to the target centralized control client, starts a timer, and stores the read / write forwarding message in the client's local storage structure. The target centralized control client then reads or writes the file to be read or written on disk. The disk feeds back the read / write operation results to the target centralized control client, which then generates a forwarding message response message and sends it to the client. If the client receives a reply within the specified time, the read-write forwarding message is deleted from the storage structure to release the cache. If the client does not receive a reply within the time limit, the read-write forwarding message is resent.

[0090] Through the embodiments of the present application, third identification information is assigned to read and write forwarding messages and managed in the client's local storage structure, achieving effective tracking of message status and automatic retransmission in abnormal situations, significantly enhancing robustness in multi-client concurrent scenarios. Even in the face of network fluctuations or target client failures, it can ensure that read and write requests are ultimately executed, and data integrity and consistency are effectively guaranteed. At the same time, the storage structure avoids disordered message management and resource waste, allowing clients to manage message queues more efficiently and reducing unnecessary message processing delays.

[0091] In an optional embodiment, in a network environment, there may be delays in the transmission of data packets, especially under wide area network (WAN) conditions, which may cause a temporary inconsistency between the information cached locally on the client and the status of the target centralized control client. In this case, even if the target centralized control client successfully performs the read and write operations, the client may fail to update the status in a timely manner, resulting in different results. Therefore, to solve this problem, in this embodiment, after forwarding the read and write forwarding message to the target centralized control client, the target centralized control client performs read and write operations on the files to be read and written on the disk, and returns a forwarding message response message to the client. The client verifies the forwarding message response message returned by the target centralized control client.

[0092] For example, a forwarding message response message may include a data checksum or digest information (e.g., an MD5 hash value). Upon receiving the forwarding message response, the client must not only check the results of the read / write operations but also compare the returned data with the data checksum in the returned result. When constructing a read / write forwarding message (e.g., a read forwarding message), the client can first hash the content to be read / written in the original metadata of the file to be read / written to obtain a local data checksum. This local data checksum is then added to the read / write forwarding message and the forwarding message containing the local data checksum is sent to the target centralized control client. The target centralized control client receives the forwarding message, performs the read / write operation, and hashes the read / write data after the operation to generate a non-local data checksum. This non-local data checksum is then encapsulated along with the read / write data in the forwarding message response message. Upon receiving the forwarding message response message within the specified timeframe, the client compares the non-local data checksum in the forwarding message response message with the local data checksum in the local cache to verify data consistency and integrity. If the verification fails—that is, the non-local data verification code in the forwarded message response does not match the locally pre-processed local data verification code—the client will mark the read / write operation as failed and resend the read / write forwarding request. At the same time, the client can also notify the target centralized control to conduct error troubleshooting to ensure the integrity of the data channel.

[0093] Through this embodiment, local data verification codes are included in read and write forwarding messages, and are compared after data processing. This not only enhances the reliability and security of data read and write operations in the distributed file system, but also optimizes the performance of the entire system, reduces the burden on key components (such as MDS), and thus improves the overall operating efficiency of the system and user experience.

[0094] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0095] According to another aspect of the embodiments of the present application, a disk file reading and writing device is also provided. The disk file reading and writing device can be used to implement a disk file reading and writing method provided in any of the above embodiments. Details that have been described are not repeated here. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0096] Figure 5 This is a structural block diagram of an optional disk file reading and writing device according to an embodiment of the present application, such as Figure 5As shown in , the disk file reading and writing device is applied to the client, and includes a first execution unit 502 , a second execution unit 504 and a third execution unit 506 .

[0097] The first execution unit 502 is configured to obtain the latest metadata information of the file to be read or written from the local cache.

[0098] The second execution unit 504 is used to construct a read-write forwarding message and determine a target centralized control client when the forwarding mark of the file to be read or written is stored in the latest metadata information; the forwarding mark is cached in the latest metadata information when multiple clients simultaneously initiate metadata open requests for the file to be read or written, and the forwarding mark indicates that the read and write requests for the file to be read or written are forwarded to the target centralized control client; the target centralized control client refers to a client that uniformly processes concurrent read and write requests from multiple clients for the file to be read or written.

[0099] The third execution unit 506 is configured to forward the read / write forwarding message to the target centralized control client, so that the target centralized control client can perform read / write operations on the to-be-read / written file in the disk.

[0100] According to an embodiment of the present application, when multiple clients initiate metadata open requests for the same file to be read or written, a forwarding tag of the file to be read or written is stored in the metadata information to obtain the latest metadata information, wherein the forwarding tag is used to instruct the client to forward the read or write request for the file to be read or written to the target centralized control client when performing subsequent read or write operations; the forwarding tag is set when the permission is updated so that when performing read or write operations, it is determined whether read or write forwarding is required based on this forwarding tag. This eliminates the need for frequent permission granting and revoking message communications with the metadata service and the need for frequent lock state switching, thereby reducing the interaction pressure between the metadata service and the client. Furthermore, by having the target centralized control client perform read and write operations on the file to be read or written, data confusion caused by concurrent operations is eliminated, so that all client read and write requests can be processed by the target centralized control client, avoiding multiple clients writing to the disk at the same time, reducing the frequent interactions between multiple clients and the metadata service, and alleviating the load on the metadata service. Therefore, the problem of high interaction pressure between the metadata service and multiple clients in the related art, resulting in slow system response time and low throughput, can be solved.

[0101] It should be noted that the first execution unit 502 in this embodiment can be used to execute the above step S202, the second execution unit 504 in this embodiment can be used to execute the above step S204, and the third execution unit 506 in this embodiment can be used to execute the above step S206.

[0102] In an exemplary embodiment, the device further includes: a fourth execution unit, configured to determine a target metadata service corresponding to the file to be read or written based on the file to be read or written; the target metadata service is configured to manage the read and write permissions of the file to be read or written; a fifth execution unit, configured to send a metadata open request to the target metadata service and receive a metadata response message fed back by the target metadata service; the metadata response message includes original metadata information of the file to be read or written and file permission information; and a sixth execution unit, configured to cache a forwarding tag in the original metadata information to obtain the latest metadata information when the file permission information indicates non-exclusive permission.

[0103] In an exemplary embodiment, the file to be read and written is assigned first identification information; the fourth execution unit includes: a first execution module, used to search a preset first local mapping table according to the first identification information to obtain a target metadata service; the first local mapping table stores the mapping relationship between different files and metadata services.

[0104] In an exemplary embodiment, the target metadata service is assigned second identification information; the second execution unit 504 includes: a second execution module, used to search a preset second local mapping table based on the second identification information to determine the target centralized control client; the second local mapping table stores the mapping relationship between different metadata services and different centralized control clients.

[0105] In an exemplary embodiment, the above-mentioned device also includes: a seventh execution unit, which is used to delete the forwarding mark from the latest metadata information when receiving a permission update message broadcast by the target metadata service, obtain new latest metadata information, and directly read the file to be read or written on the disk; the permission update message indicates that the file permission of the file to be read or written is switched from non-exclusive permission to exclusive permission.

[0106] In an exemplary embodiment, the read-write forwarding message includes at least one of the following: a read forwarding message and a write forwarding message; the second execution unit 504 also includes: a third execution module, which is used to store the forwarding mark of the file to be read and written in the latest metadata information, and the read-write forwarding message indicates that a read operation is performed on the file to be read and written, and fill in the first preset field in the first forwarding message to obtain a read forwarding message; the first preset field includes the first identification information of the file to be read and written, the read operation type, the read starting offset address, the read data length and the memory address for storing the read data content; the fourth execution module, which is used to store the forwarding mark of the file to be read and written in the latest metadata information, and the read-write forwarding message indicates that a write operation is performed on the file to be read and written, and fill in the second preset field in the second forwarding message to obtain a write forwarding message; the second preset field includes the first identification information, the write operation type, the write starting offset address, the write data length and the byte stream of the data content to be written.

[0107] In an exemplary embodiment, the read-write forwarding message is assigned with third identification information; the above-mentioned device also includes: a first saving unit, used to save the read-write forwarding message and the third identification information into a storage structure, waiting for the forwarding message response message of the target centralized control client; a first sending unit, used to resend the read-write forwarding message if no forwarding message response message is received within a timeout; an eighth execution unit, used to delete the read-write forwarding message and the third identification information from the storage structure if a forwarding message response message is received within a specified time range.

[0108] According to another aspect of the embodiments of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute the steps of any of the above-described method embodiments through the computer program. In an exemplary embodiment, the electronic device may further comprise a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0109] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0110] According to another aspect of the embodiments of the present application, a computer program product is provided. The computer program product includes a computer program / instruction. The computer program / instruction includes a program code for executing the method shown in the flowchart.

[0111] Figure 6 The following schematically shows a block diagram of a computer system structure of an electronic device for implementing an embodiment of the present application. Figure 6 As shown, computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for system operation. CPU 601, ROM 602, and RAM 603 are interconnected via bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0112] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, mouse, and the like; an output section 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a local area network card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. Removable media 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read from the removable media can be installed in the storage section 608 as needed.

[0113] In particular, according to an embodiment of the present application, the processes described in the various method flow charts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit 601, the various functions provided by the embodiments of the present application are executed. The above-mentioned serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0114] It should be noted that Figure 6 The computer system 600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0115] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices, they can be implemented using program code executable by the computing device, and thus, they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0116] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for reading and writing disk files, characterized in that: Applied to the client, including: Get the latest metadata information of the file to be read or written from the local cache; In the case where the forwarding mark of the file to be read and written is stored in the latest metadata information, a read and write forwarding message is constructed and a target centralized control client is determined; the forwarding mark is cached in the latest metadata information when multiple clients simultaneously initiate metadata open requests for the file to be read and written, and the forwarding mark indicates that the read and write requests for the file to be read and written are forwarded to the target centralized control client; the target centralized control client refers to a client that uniformly processes concurrent read and write requests for the file to be read and written by the multiple clients; the target centralized control client is bound to a target metadata service, and the target metadata service is used to manage the read and write permissions of the file to be read and written; the read and write forwarding message of the file to be read and written belonging to the target metadata service is sent to the target centralized control client for processing; the target centralized control client serially completes data reading and writing on the disk in the order of the read and write requests, and replies the read and write execution results and the read data to the client; The read-write forwarding message is forwarded to the target centralized control client, so that the target centralized control client performs read and write operations on the to-be-read-written file in the disk.

2. The method according to claim 1, characterized in that Before obtaining the latest metadata information of the file to be read or written from the local cache, the method further includes: Determining the target metadata service corresponding to the file to be read or written according to the file to be read or written; Sending an open metadata request to the target metadata service and receiving a metadata response message fed back by the target metadata service; the metadata response message includes the original metadata information of the file to be read or written and file permission information; In a case where the file permission information represents non-exclusive permission, the forwarding tag is cached in the original metadata information to obtain the latest metadata information.

3. The method according to claim 2, characterized in that The file to be read or written is assigned with first identification information; and determining a target metadata service corresponding to the file to be read or written according to the file to be read or written includes: According to the first identification information, a preset first local mapping table is searched to obtain the target metadata service; the first local mapping table stores mapping relationships between different files and metadata services.

4. The method according to claim 2, characterized in that The target metadata service is assigned with second identification information; and the determining of the target centralized control client includes: According to the second identification information, a preset second local mapping table is searched to determine the target centralized control client; the second local mapping table stores mapping relationships between different metadata services and different centralized control clients.

5. The method according to claim 2, characterized in that After forwarding the read-write forwarding message to the target centralized control client, the method further includes: Upon receiving a permission update message broadcast by the target metadata service, the forwarding flag is deleted from the latest metadata information, new latest metadata information is obtained, and the file to be read or written on the disk is directly read; the permission update message indicates that the file permission of the file to be read or written is switched from non-exclusive permission to exclusive permission.

6. The method according to claim 1, characterized in that The read-write forwarding message includes at least one of the following: a read forwarding message and a write forwarding message; when the forwarding mark of the file to be read or written is stored in the latest metadata information, constructing the read-write forwarding message includes: In a case where a forwarding tag of the file to be read or written is stored in the latest metadata information and the read / write forwarding message indicates a read operation on the file to be read or written, a first preset field is filled in the first forwarding message to obtain the read forwarding message; the first preset field includes first identification information of the file to be read or written, a read operation type, a read start offset address, a read data length, and a memory address for storing the read data content; When the forwarding mark of the file to be read or written is stored in the latest metadata information and the read / write forwarding message represents a write operation on the file to be read or written, a second preset field is filled in the second forwarding message to obtain the write forwarding message; the second preset field includes the first identification information, the write operation type, the write start offset address, the write data length and the data content byte stream to be written.

7. The method according to claim 1, characterized in that The read-write forwarding message is assigned third identification information; after forwarding the read-write forwarding message to the target centralized control client, the method further includes: Saving the read-write forwarding message and the third identification information to a storage structure, and waiting for a forwarding message response message from the target centralized control client; If no forwarding message response message is received within a timeout period, resending the read-write forwarding message; When the forwarding message response message is received within a specified time range, the read-write forwarding message and the third identification information are deleted from the storage structure.

8. A disk file reading and writing device, characterized in that: Applied to the client, including: The first execution unit is configured to obtain the latest metadata information of the file to be read or written from the local cache; The second execution unit is used to construct a read-write forwarding message and determine a target centralized control client when a forwarding mark of the file to be read or written is stored in the latest metadata information; the forwarding mark is cached in the latest metadata information when multiple clients simultaneously initiate metadata open requests for the file to be read or written, and the forwarding mark indicates that the read and write requests for the file to be read or written are forwarded to the target centralized control client; the target centralized control client refers to a client that uniformly processes concurrent read and write requests of the multiple clients for the file to be read or written; the target centralized control client is bound to a target metadata service, and the target metadata service is used to manage the read and write permissions of the file to be read or written; the read and write forwarding message of the file to be read or written belonging to the target metadata service is sent to the target centralized control client for processing; the target centralized control client serially completes data reading and writing on disk in the order of the read and write requests, and replies the read and write execution results and the read data to the client; The third execution unit is configured to forward the read-write forwarding message to the target centralized control client, so that the target centralized control client performs read and write operations on the to-be-read / written file in the disk.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.

Images