A parallel file storage system and method

Through parallel file storage optimization modules, combined with parallel memory, layered storage and buffer modules, the long access cycle and read errors caused by dual-port memory are solved, and efficient parallel file storage and fast access is achieved. It is suitable for I /O Frequent enterprise applications.

CN114816260BActive Publication Date: 2024-11-05GUANGDONG AOFEI DATA TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210513731.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-11
Publication Date
2024-11-05
Estimated Expiration
2042-05-11

AI Technical Summary

Technical Problem

The existing parallel file storage system uses dual-port memory to cause a long access cycle, resulting in slower storage speed of parallel file and prone to read errors.

Method used

Adopting parallel file storage optimization modules, including parallel memory modules, layered storage modules and add buffer modules, by increasing the memory word length and number of memory cells, combining cross-access and conflict-free access memory, optimizes the storage structure and buffer design , improve storage efficiency and reliability.

Benefits of technology

It improves the speed and efficiency of parallel file storage, reduces read errors, and can be used as a high-performance storage system for enterprise application environments with frequent I/O access, avoids storage blockage, and enhances the computing power of the server.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114816260B_ABST
    Figure CN114816260B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of file storage technology, specifically to a parallel file storage system and method, comprising a parallel file storage system, wherein the output end of the parallel file storage system is electrically connected to a parallel file storage optimization module, one output end of the parallel file storage optimization module is electrically connected to a parallel memory module, and the other output end of the parallel file storage optimization module is electrically connected to a layered storage module, and the other output end of the parallel file storage optimization module is electrically connected to an added buffer module, and the cross-access memory comprises a low-order cross module and a high-order cross module, and the continuous addresses of the cross-access parallel memory in the low-order cross module are distributed in adjacent different modules, so that multi-module parallel access can be realized, and the memory bandwidth can be improved, and the access addresses in the high-order cross program are sequentially allocated to each module, and when a certain module is working, other modules are not working, so as to regulate and control the parallel file storage.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The present invention relates to the technical field of file storage, and in particular to a parallel file storage system and method. Background Art

[0002] With the continuous expansion of computer application fields, the amount of information processed is increasing, and the I / O devices of modern computers are also increasing. Therefore, improving the speed of memory access has become an urgent task, and a large number of files need to be stored in computer terminals.

[0003] The parallel file system is an optimized high-performance file system provided by the object storage service. It provides millisecond-level access latency, TB / s-level bandwidth, and millions of IOPS. It can quickly process high-performance computing (HPC) workloads. The speed between the CPU and storage is often mismatched, and data access often becomes a computing bottleneck. This problem can be solved by increasing the speed on the one hand, and optimizing the storage structure on the other.

[0004] Most of the current parallel file storage systems and methods use dual-port memories to achieve the purpose of parallel file storage, but the storage access cycle is long, resulting in slow parallel file storage and frequent read errors during the reading process.

[0005] To this end, a parallel file storage system and method are proposed. Summary of the invention

[0006] The object of the present invention is to provide a parallel file storage system and method to solve the problem mentioned in the above background technology that most of the current parallel file storage systems and methods use dual-port memories to achieve the purpose of parallel file storage, but the storage access cycle is long, resulting in slow parallel file storage and frequent reading errors during the reading process.

[0007] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a parallel file storage system and method, comprising a parallel file storage module, the output end of the parallel file storage system is electrically connected to a parallel file storage optimization module, one output end of the parallel file storage optimization module is electrically connected to a parallel memory module, and the other output end of the parallel file storage optimization module is electrically connected to a hierarchical storage module, and the other output end of the parallel file storage optimization module is electrically connected to an additional buffer module, one output end of the parallel memory module is electrically connected to a memory word length increase module, and the other output end of the parallel memory module is electrically connected to a storage body number increase module, and the output ends of the memory word length increase module and the storage body number increase module are electrically connected to a parallel access memory.

[0008] Preferably, the parallel memory modules are divided into space parallel type and time parallel type.

[0009] Preferably, a dual-port memory is provided at one end of the parallel memory module, and a multi-memory module parallel memory is provided at the other end of the parallel memory module.

[0010] Preferably, the multi-storage module parallel storage is composed of multiple storage bodies.

[0011] Preferably, the other output end of the added buffer module is electrically connected to various buffers.

[0012] Preferably, the various buffers are set as general registers.

[0013] Preferably, the storage body number increasing module includes a cross-access memory and a conflict-free access memory.

[0014] Preferably, the cross-access memory includes a low-order cross-access module and a high-order cross-access module.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] 1. A parallel file storage module, wherein the output end of the parallel file storage system is electrically connected to a parallel file storage optimization module, one output end of the parallel file storage optimization module is electrically connected to a parallel memory module, and the other output end of the parallel file storage optimization module is electrically connected to a hierarchical storage module, and the other output end of the parallel file storage optimization module is electrically connected to an additional buffer module, one output end of the parallel memory module is electrically connected to a memory word length increase module, and the other output end of the parallel memory module is electrically connected to a memory body number increase module, the output ends of the memory word length increase module and the memory body number increase module are electrically connected to a parallel access memory, and the parallel memory module is divided into a spatial parallel type and a temporal parallel type. Parallel type, one end of the parallel memory module is provided with a dual-port memory, and the other end of the parallel memory module is provided with a multi-memory module parallel memory, the multi-memory module parallel memory is composed of a plurality of memory bodies, the other end of the output of the added buffer module is electrically connected to various buffers, and the various buffers are set as general registers. The memory body number increase module includes a cross-access memory and a conflict-free access memory, and the cross-access memory includes a low-order cross-module and a high-order cross-module. The parallel memory module, the hierarchical storage module and the added buffer module are used as the main method for optimizing the storage in the parallel file storage optimization module, and the parallel memory module is used to increase the storage access of the parallel file in advance. In addition, by using layered storage modules and adding buffer modules at the branch, the online data in the layered storage module is further layered, and the data information storage is guaranteed under a more scientific concept. It can be used as an independent high-performance storage system for enterprise application environments with frequent I / O access, and the added buffer module connected in series can be used to transmit and temporarily store data, participate in arithmetic and logical operations, and save the results of operations, greatly increasing the computing power of the server and avoiding storage congestion. In addition, in conjunction with the memory word length increase module and the storage body number increase module in the parallel memory module, the storage access volume is quickly increased, the parallel access memory is improved, the storage efficiency of parallel files is increased, and in conjunction with the storage The number of banks increases by using cross-access memory and conflict-free access memory in the module. The odd and even addresses of the bank are separated by cross-access memory to compensate for the relatively slow read and write speeds of DRAM and other memories. Cross-storage disperses continuous information into various blocks, thereby improving the read and write storage efficiency. The cross-access memory includes a low-order cross-module and a high-order cross-module. Through the cross-access parallel memory in the low-order cross-module, continuous addresses are distributed in adjacent different modules, which can realize multi-module parallel access and improve memory bandwidth. The access addresses in the high-order cross program are sequentially allocated to each module. When a module is working, other modules are not working, and parallel file storage is regulated. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a diagram of the central control system of the present invention;

[0018] Figure 2 A system diagram of a parallel memory module of the present invention;

[0019] Figure 3 A module system diagram for increasing the number of storage bodies of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0022] See also Figures 1 to 3 , the present invention provides a parallel file storage system and method technical solution:

[0023] A parallel file storage module, the output end of the parallel file storage system is electrically connected to a parallel file storage optimization module, one end of the output of the parallel file storage optimization module is electrically connected to a parallel memory module, and the other end of the output of the parallel file storage optimization module is electrically connected to a hierarchical storage module, and the other end of the output of the parallel file storage optimization module is electrically connected to an added buffer module, one end of the output of the parallel memory module is electrically connected to a memory word length increase module, and the other end of the output of the parallel memory module is electrically connected to a memory body number increase module, the output ends of the memory word length increase module and the memory body number increase module are electrically connected to a parallel access memory, and the parallel memory module is divided into a spatial parallel type and a temporal parallel type. Type, one end of the parallel memory module is provided with a dual-port memory, and the other end of the parallel memory module is provided with a multi-memory module parallel memory, the multi-memory module parallel memory is composed of a plurality of memory bodies, the other end of the output of the added buffer module is electrically connected to various buffers, and the various buffers are set as general registers. The memory body number increase module includes a cross-access memory and a conflict-free access memory, and the cross-access memory includes a low-order cross-module and a high-order cross-module. The parallel memory module, the hierarchical storage module and the added buffer module are used as the main method for optimizing the storage in the parallel file storage optimization module, and the parallel memory module is used to increase the storage access of the parallel file in advance. In addition By adopting hierarchical storage modules and adding buffer modules at the branch, the online data in the hierarchical storage module is further hierarchical. Under a more scientific concept, the data information storage is guaranteed. It can be used as an independent high-performance storage system for enterprise application environments with frequent I / O access. The added buffer module connected in series can be used to transmit and temporarily store data, participate in arithmetic and logical operations, and save the operation results, which greatly increases the computing power of the server and avoids storage congestion. In addition, in conjunction with the memory word length increase module and the storage body number increase module in the parallel memory module, the storage access volume is quickly increased, the parallel access memory is improved, the storage efficiency of parallel files is increased, and in conjunction with the storage body number increase module, the storage word length increase module and the storage body number increase module in the parallel memory module are used to increase the storage word length increase module and the storage body number increase module in the parallel memory module. The cross-access memory and conflict-free access memory in the number-increasing module are used to separate the odd and even address parts of the storage body through the cross-access memory to compensate for the relatively slow read and write speeds of memories such as DRAM. The cross-storage disperses continuous information into various blocks, thereby improving the read and write storage efficiency. The cross-access memory includes a low-order cross-module and a high-order cross-module. Through the cross-access parallel memory in the low-order cross-module, the continuous addresses are distributed in different adjacent modules, which can realize multi-module parallel access and improve the memory bandwidth. The access addresses in the high-order cross program are sequentially assigned to each module. When a module is working, other modules are not working, and parallel file storage is regulated.

[0024] Usage: When users perform parallel file storage, they will first transfer and control the file to the parallel file storage optimization module, and use the parallel memory module to increase the storage access of parallel files. In addition, by using the layered storage module and adding the buffer module at the branch, the online data in the layered storage module is further layered. Under a more scientific concept, the storage of data information is guaranteed. It can be used as an independent high-performance storage system for enterprise application environments with frequent I / O access. The added buffer module connected in series can be used to transmit and temporarily store data, participate in arithmetic and logical operations, and save the results of operations, greatly increasing the computing power of the server and avoiding storage congestion. Through the cross-access parallel memory continuous addresses in the low-order cross module, the continuous addresses are distributed in different adjacent modules, which can realize multi-module parallel access and improve the memory bandwidth. The access addresses in the high-order cross program are sequentially allocated to each module. When a module is working, other modules do not work, and parallel file storage is regulated. In addition, with the memory word length increase module and the storage body number increase module in the parallel memory module, the storage access volume is quickly increased, the parallel access memory is improved, and the storage efficiency of parallel files is increased.

[0025] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A parallel file storage system, comprising a parallel file storage module, wherein the output end of the parallel file storage module is electrically connected to a parallel file storage optimization module, one output end of the parallel file storage optimization module is electrically connected to a parallel memory module, and the other output end of the parallel file storage optimization module is electrically connected to a hierarchical storage module, and the other output end of the parallel file storage optimization module is electrically connected to an additional buffer module, the parallel memory module is divided into a spatial parallel type and a temporal parallel type, one output end of the parallel memory module is electrically connected to a memory word length increasing module, and the other output end of the parallel memory module is electrically connected to a memory body number increasing module, the memory word length increasing module and the memory body number increasing module are electrically connected to each other. The output end of the module is electrically connected to a parallel access memory. The storage body number increasing module includes a cross access memory and a conflict-free access memory. The cross access memory includes a low-order cross module and a high-order cross module. The odd address and even address parts of the storage body are separated by the cross access memory. The cross storage disperses the continuous information into each block. The continuous addresses are distributed into different adjacent modules by the cross access parallel memory in the low-order cross module. The access addresses in the high-order cross program are sequentially allocated to each module. A dual-port memory is provided at one end of the parallel memory module, and a multi-storage module parallel memory is provided at the other end of the parallel memory module. The multi-storage module parallel memory consists of multiple storage bodies.

2. A parallel file storage system according to claim 1, characterized in that: The other output end of the added buffer module is electrically connected to various buffers.

3. A parallel file storage system according to claim 2, characterized in that: The various buffers are set as general registers.