Method, system and device for improving data access performance and storage medium
By caching inode information at the application layer and optimizing the file system's read and write processes, the MDS performance bottleneck problem in the distributed file system is resolved, achieving more efficient data access performance and reducing hardware costs.
Patent Information
- Application Number
- CN202510722684.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-30
- Publication Date
- 2025-09-16
AI Technical Summary
In existing distributed file systems, as the number of files increases, the metadata server (MDS) becomes a performance bottleneck. Although increasing the number of MDS servers alleviates the load, it brings management complexity, resource consumption and cost issues.
By caching inode information at the application layer, the inode information is stored in the database when writing files, and is directly read from the OSD when reading files, reducing interaction with the MDS and optimizing the read and write processes of the file system.
It effectively alleviates MDS performance bottlenecks, reduces MDS load, improves system performance and scalability, reduces hardware and maintenance costs, and improves file system stability and response time.
Smart Images
Figure CN120653858A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of big data technology, and specifically to a method, system, device, and storage medium for improving data access performance. Background Art
[0002] As a key component in modern cloud computing and big data environments, distributed file systems (DFSs) are tasked with storing large amounts of data across multiple servers and automatically distributing and recovering it, ensuring high performance and reliability. However, with the rapid growth of data volumes, DFSs face numerous challenges, particularly in scenarios with a large number of files and a wide directory structure. The performance bottleneck of metadata servers (MDSs) is particularly prominent.
[0003] The Metadata Server (MDS) is a key component in distributed file systems such as CephFS. It manages and maintains the file system's metadata, including key information such as file names, permissions, and directory structures. In a distributed file system, when a client sends a read or write request, the MDS must quickly locate the required data and provide a corresponding response. However, as file system usage increases, a single MDS server can become a performance bottleneck for the entire system due to the high volume of metadata requests it handles.
[0004] In existing distributed file system technologies, some solutions attempt to improve system performance and scalability by increasing the number of MDS servers to distribute the load. For example, in the Ceph file system, additional MDS servers can be added to the Ceph cluster and configured as MDSs to share the load of existing MDS servers. These new MDS servers will begin receiving and processing metadata requests from clients, reducing the pressure on individual MDS servers. Furthermore, load balancing algorithms can be implemented to distribute client requests as evenly as possible across multiple MDS servers, ensuring that each MDS server can perform to its full potential.
[0005] However, while increasing the number of MDS servers can alleviate performance bottlenecks to some extent, it also introduces new challenges. First, as the number of MDS servers increases, metadata management and synchronization become more complex, requiring additional mechanisms to ensure data consistency and reliability. Second, the addition of MDS servers also brings additional costs and resource consumption, including hardware costs, maintenance costs, and energy consumption. Summary of the Invention
[0006] To this end, embodiments of the present invention provide a method, system, device, and storage medium for improving data access performance to address the MDS performance bottleneck problem in the prior art when there are too many files and a wide directory structure in a distributed file system.
[0007] In order to achieve the above objectives, the embodiments of the present invention provide the following technical solutions:
[0008] According to a first aspect of an embodiment of the present invention, a method for improving data access performance is provided. The method is applied to an application layer, wherein the application layer writes a file to a distributed file system through a client and stores the file inode information returned by the client in a database, specifically comprising:
[0009] Receive a data operation request from a client and determine whether the data operation request is to write a file;
[0010] If the data operation request is to write a file, the file to be written will be written to the distributed file system, and the inode information returned by the client will be cached in the database;
[0011] If the data operation request is to read a file, the inode information of the file to be read is queried from the database, the file is directly read from the OSD using the inode information, and the file is returned to the application layer.
[0012] Furthermore, the inode information includes inode number, file size, stripe_unit, strip_count, object_size, pool_id, and pool_ns;
[0013] Among them, stripe_unit, strip_count, object_size, pool_id, and pool_ns are file layout information.
[0014] Furthermore, if the data operation request is to write a file, the file to be written is written to the distributed file system, and the inode information returned by the client is cached in the database, including:
[0015] The business layer application in the application layer sends the file to be written to the client, and the client sends a write file request to the MDS;
[0016] The MDS obtains the inode information of the file to be written and sends it to the client. The client sends the file to be written to the OSD for writing. After the file is successfully written, the file inode information is returned to the client.
[0017] The client sends the received inode information to the business layer application, and the business layer application saves the inode information in the database.
[0018] Furthermore, if the data operation request is to read a file, the inode information of the file to be read is queried from the database, the file is directly read from the OSD using the inode information, and the file is returned to the application layer, including:
[0019] Obtain the inode information of the file to be read from the database and send the inode information to the business layer application;
[0020] Get the location of the file to be read using the inode information as a parameter and send the location information to the client;
[0021] The client uses the location information to read the file from the OSD and returns the file data to the client;
[0022] The client sends the file data to the business layer application to complete the file reading.
[0023] Furthermore, the application scenario of the method is write-once-read-many.
[0024] According to a second aspect of an embodiment of the present invention, a system for improving data access performance is provided, the system comprising:
[0025] A data operation request receiving module is used to receive a data operation request from a client and determine whether the data operation request is to write a file;
[0026] The data writing module is used to perform the following steps:
[0027] If the data operation request is to write a file, the file to be written will be written to the distributed file system, and the inode information returned by the client will be cached in the database;
[0028] The data reading module is used to perform the following steps:
[0029] If the data operation request is to read a file, the inode information of the file to be read is queried from the database, the file is directly read from the OSD using the inode information, and the file is returned to the application layer.
[0030] According to a third aspect of an embodiment of the present invention, there is provided a device for improving data access performance, the device comprising: a processor and a memory;
[0031] The memory is used to store one or more program instructions;
[0032] The processor is configured to run one or more program instructions to execute the steps of a method for improving data access performance as described in any one of the above items.
[0033] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for improving data access performance as described in any one of the above items are implemented.
[0034] The embodiments of the present invention have the following advantages:
[0035] The embodiment of the present invention effectively alleviates the performance bottleneck problem of MDS in the distributed file system by optimizing the process and method of reading and writing files by the application layer and the client, and improves the overall performance and scalability of the system. After the file is successfully written, the application layer needs to save the file inode information to the database cache; in the file reading operation, the application layer passes the cached file inode information to the file system client. The file system client no longer communicates with the MDS to obtain the file inode information, but reads the file directly from the OSD, effectively reducing the load of the MDS and shortening the file reading response time. In the case of intensive IO requests, it can reduce the delay of client requests and avoid business failures caused by the accumulation of a large number of businesses, greatly improving the stability of the file system, and shortening the response time. It can meet business needs with high response time requirements, reduce the number of MDS servers deployed, and reduce hardware investment costs and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.
[0037] The structures, proportions, sizes, etc. illustrated in this specification are intended only to complement the contents disclosed herein and to facilitate understanding and reading by persons familiar with the art. They are not intended to limit the conditions under which the present invention may be implemented and therefore have no substantive technical significance. Any structural modifications, changes in proportions, or adjustments in sizes, without affecting the efficacy and objectives of the present invention, shall still fall within the scope of the technical contents disclosed herein.
[0038] Figure 1 A schematic diagram of the logical structure of a system for improving data access performance provided by an embodiment of the present invention;
[0039] Figure 2 A flowchart of a method for improving data access performance provided by an embodiment of the present invention;
[0040] Figure 3 A schematic diagram of a file writing process in a method for improving data access performance provided by an embodiment of the present invention;
[0041] Figure 4 A schematic diagram of a file reading process in a method for improving data access performance provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0042] The following describes the implementation of the present invention using specific embodiments. Those skilled in the art will readily understand the other advantages and benefits of the present invention from the disclosure herein. Obviously, the embodiments described are only a portion of the present invention, not all of it. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.
[0043] In the field of distributed file systems, in order to solve the performance bottleneck problem of MDS, existing technologies have proposed a series of detailed technical solutions, aiming to meet the needs of large-scale file storage by enhancing the processing capabilities of MDS, optimizing load balancing strategies, and improving metadata synchronization efficiency.
[0044] First, enhancing the processing capabilities of MDS is a key step. Some technologies employ a horizontal scaling strategy, increasing the number of MDS servers to distribute the load. For example, by deploying multiple MDS instances, each responsible for handling a specific range of metadata requests, parallel processing and load balancing are achieved. Furthermore, to improve the performance of MDS servers, high-performance hardware and dedicated storage devices, such as SSDs, are employed to accelerate metadata read and write operations. Furthermore, a caching mechanism is introduced to cache frequently accessed metadata in memory, reducing disk accesses and improving response times.
[0045] Secondly, optimizing load balancing strategies is crucial for improving the overall performance of MDS. Existing technologies employ a variety of load balancing algorithms, such as round-robin, minimum connections, and weighted round-robin, to evenly distribute client requests across multiple MDS servers. Furthermore, intelligent load balancing technology has been introduced to dynamically adjust request distribution strategies based on the real-time load of MDS servers, ensuring that each MDS server operates within its processing capacity, avoiding overload and performance degradation.
[0046] Existing technologies have also undergone a series of optimizations for metadata synchronization. Because multiple MDS servers in a distributed file system need to share and synchronize metadata to ensure data consistency and reliability, distributed locks, transaction mechanisms, and message queues are used to coordinate operations between MDS servers. These technologies ensure the secure transmission and update of metadata across multiple MDS servers, while preventing data conflicts and loss.
[0047] Although existing technologies have achieved certain results in alleviating the MDS performance bottleneck in distributed file systems, they still have some obvious shortcomings and deficiencies.
[0048] First, regarding scalability, while increasing the number of MDS servers can help distribute the load, it also introduces management and maintenance complexity. As the number of servers increases, metadata management and synchronization become more difficult, requiring additional mechanisms and resources to ensure data consistency and reliability. Furthermore, an excessive number of MDS servers increases system complexity and costs, including hardware, maintenance, and energy consumption, which is a significant burden for large-scale distributed file systems.
[0049] Secondly, in terms of load balancing, while existing technologies employ various algorithms to optimize request distribution strategies, they still have limitations in practice. For example, some algorithms may not accurately reflect the real-time load of MDS servers, leading to uneven request distribution, with some servers overloaded and others idle. This not only degrades overall system performance but also wastes resources.
[0050] Furthermore, metadata synchronization is a challenge in existing technologies. Because multiple MDS servers in a distributed file system need to share and synchronize metadata, ensuring data consistency and reliability is crucial. However, in practice, metadata synchronization often encounters latency and inconsistency issues, impacting system performance and stability. For example, if an MDS server fails, other servers may need to wait a long time to synchronize with the latest metadata, resulting in service interruptions or data inconsistencies.
[0051] Although existing technologies have provided some solutions, they still have many challenges and shortcomings. In order to solve the MDS performance bottleneck problem when there are too many files and a wide directory structure in the distributed file system,
[0052] refer to Figure 1 An embodiment of the present invention discloses a system for improving data access performance. The system includes: a data operation request receiving module 1; a data writing module 2; and a data reading module 3.
[0053] Corresponding to the above-disclosed system for improving data access performance, an embodiment of the present invention further discloses a method for improving data access performance. The following describes in detail the method for improving data access performance disclosed in an embodiment of the present invention in conjunction with the above-described system for improving data access performance.
[0054] In the process of reading and writing files in a distributed file system, the technical solution of the existing technology is that the client will first communicate with the metadata server (MDS) to obtain the metadata information of the file to be read and written, and then connect to the OSD to perform the file read and write operations. In the scenario of one write and multiple reads, once a file in the distributed file system is written, it will no longer be modified, and the file's inode (inode number, file size, stripe_unit, strip_count, object_size, pool_id, pool_ns) information will remain unchanged, where stripe_unit, strip_count, object_size, pool_id, and pool_ns are file layout information. (In subsequent documents and design drawings, inode information refers to inode number, file size, stripe_unit, strip_count, object_size, pool_id, and pool_ns information.) If the inode information is cached, when reading a file, the client can read the file directly from the OSD based on the cached inode information.
[0055] refer to Figure 2 In view of the fact that the file inode information remains unchanged in a one-write-multiple-read scenario, the present invention discloses a method for improving data access performance. The method is applied to the application layer, and the application layer writes a file to a distributed file system through a client and stores the file inode information returned by the client in a database. The method specifically includes: receiving a data operation request from the client and determining whether the data operation request is to write a file; if the data operation request is to write a file, writing the file to be written to the distributed file system, and caching the inode information returned by the client in a database; if the data operation request is to read a file, querying the inode information of the file to be read from the database, using the inode information to directly read the file from the OSD, and returning the file to the application layer.
[0056] Furthermore, the inode information includes inode number, file size, stripe_unit, strip_count, object_size, pool_id, and pool_ns.
[0057] Among them, stripe_unit, strip_count, object_size, pool_id, and pool_ns are file layout information.
[0058] The client has two main functions: writing files and returning file inode information, and reading files using file inode information. The application layer writes files to the distributed file system through the file system client and stores the file inode information returned by the file system client in the database.
[0059] Further, refer to Figure 3 If the data operation request is to write a file, the file to be written will be written to the distributed file system, and the inode information returned by the client will be cached in the database, including: the business layer application in the application layer sends the file to be written to the client, and the client sends a write file request to the MDS; the MDS obtains the inode information of the file to be written and sends it to the client, and the client sends the file to be written to the OSD to perform the write operation, and returns the file inode information to the client after the file is successfully written; the client sends the received inode information to the business layer application, and the business layer application saves the inode information in the database.
[0060] When a file needs to be read, the application first queries the database for the file's inode information. Since the file's inode information is known, the file system client no longer needs to request this information from the metadata server (MDS) to determine the file's location. Instead, it uses this inode information to read the file directly from the OSD and return the data to the application.
[0061] Further, refer to Figure 4 If the data operation request is to read a file, the inode information of the file to be read is queried from the database, the file is directly read from the OSD using the inode information, and the file is returned to the application layer, including: obtaining the inode information of the file to be read from the database and sending the inode information to the business layer application; obtaining the location of the file to be read using the inode information as a parameter and sending the location information to the client; the client uses the location information to read the file from the OSD and return the file data to the client; the client sends the file data to the business layer application to complete the file reading.
[0062] Furthermore, the application scenario of the method is one-write-many-reads. In the embodiment of the present invention, the inode information is cached by the application layer, and the file system client also has the function of reading files through the inode information.
[0063] For systems with relatively intensive IO requests, the embodiments of the present invention can reduce the latency of client requests, avoid service failures caused by the accumulation of a large number of services, and greatly improve the stability of the file system.
[0064] Furthermore, the solution proposed in the embodiment of the present invention has a shorter response time and can meet business needs with very high response time requirements.
[0065] By utilizing the embodiments of the present invention, the number of deployed MDS servers can be reduced, and hardware investment costs and maintenance costs can be lowered.
[0066] The embodiments of the present invention have the following advantages:
[0067] 1) When the distributed file system has a large number of files and a large file width, it can effectively solve the problem of service failure caused by long client delays;
[0068] 2) For high-concurrency, low-latency business scenarios, there is no need to increase the number of additional MDS servers to meet business needs.
[0069] In addition, an embodiment of the present invention also provides a device for improving data access performance, the device comprising: a processor and a memory; the memory is used to store one or more program instructions; the processor is used to run one or more program instructions to execute the steps of a method for improving data access performance as described in any of the above items.
[0070] In addition, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for improving data access performance as described in any one of the above items are implemented.
[0071] In the embodiments of the present invention, the processor may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0072] The methods, steps, and logic diagrams disclosed in the embodiments of the present invention can be implemented or executed. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in a storage medium well-established in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The processor reads the information from the storage medium and, in conjunction with its hardware, completes the steps of the aforementioned methods.
[0073] The storage medium may be a memory and may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.
[0074] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.
[0075] Volatile memory may be random access memory (RAM), which is used as an external cache memory. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM).
[0076] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.
[0077] Those skilled in the art will appreciate that in one or more of the above examples, the functions described herein can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media includes any medium that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.
[0078] Although the present invention has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications and improvements may be made thereto. Therefore, such modifications and improvements, without departing from the spirit of the present invention, are intended to be within the scope of protection claimed herein.
Claims
1. A method for improving data access performance, characterized in that: The method is applied to the application layer, which writes files into the distributed file system through the client and stores the file inode information returned by the client into the database, specifically including: Receive a data operation request from a client and determine whether the data operation request is to write a file; If the data operation request is to write a file, the file to be written will be written to the distributed file system, and the inode information returned by the client will be cached in the database; If the data operation request is to read a file, the inode information of the file to be read is queried from the database, the file is directly read from the OSD using the inode information, and the file is returned to the application layer.
2. A method for improving data access performance according to claim 1, characterized in that: The inode information includes inode number, file size, stripe_unit, strip_count, object_size, pool_id, and pool_ns; Among them, stripe_unit, strip_count, object_size, pool_id, and pool_ns are file layout information.
3. A method for improving data access performance according to claim 2, characterized in that: If the data operation request is to write a file, the file to be written is written to the distributed file system, and the inode information returned by the client is cached in the database, including: The business layer application in the application layer sends the file to be written to the client, and the client sends a write file request to the MDS; The MDS obtains the inode information of the file to be written and sends it to the client. The client sends the file to be written to the OSD for writing. After the file is successfully written, the file inode information is returned to the client. The client sends the received inode information to the business layer application, and the business layer application saves the inode information in the database.
4. A method for improving data access performance according to claim 3, characterized in that: If the data operation request is to read a file, the inode information of the file to be read is queried from the database, the file is directly read from the OSD using the inode information, and the file is returned to the application layer, including: Obtain the inode information of the file to be read from the database and send the inode information to the business layer application; Get the location of the file to be read using the inode information as a parameter and send the location information to the client; The client uses the location information to read the file from the OSD and returns the file data to the client; The client sends the file data to the business layer application to complete the file reading.
5. A method for improving data access performance according to claim 4, characterized in that: The application scenario of the method is one-write-many-reads.
6. A system for improving data access performance, characterized in that: The system comprises: A data operation request receiving module is used to receive a data operation request from a client and determine whether the data operation request is to write a file; The data writing module is used to perform the following steps: If the data operation request is to write a file, the file to be written will be written to the distributed file system, and the inode information returned by the client will be cached in the database; The data reading module is used to perform the following steps: If the data operation request is to read a file, the inode information of the file to be read is queried from the database, the file is directly read from the OSD using the inode information, and the file is returned to the application layer.
7. A device for improving data access performance, characterized in that: The device includes: a processor and a memory; The memory is used to store one or more program instructions; The processor is configured to run one or more program instructions to execute the steps of a method for improving data access performance as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for improving data access performance as claimed in any one of claims 1 to 5.