Data storage structure optimization method and system and medium

Abstract

The invention discloses a data storage structure optimization method, which comprises the following steps: creating a first response area, creating a quick cache area and a common storage area, when a server cluster receives a data reading request, the first response area responding, and if the first response area does not have data corresponding to the reading request, enabling the quick cache area to response; if the server cluster receives data corresponding to the reading request, the quick cache area makes a response, if the quick cache area does not have data corresponding to the reading request, the common storage area makes a response, and when the server cluster receives the data, the first response area caches the data and asynchronizes the data to the common storage area and the quick cache area. The distributed storage structure can be optimized, the flexibility of data writing and reading of the server cluster is improved, the writing load of the server cluster is reduced, the bandwidth of data reading is improved, and an excellent working space and an excellent working state are provided for an application program.

Description

technical field

[0001] The invention relates to the field of data storage, in particular to a method, system and medium for optimizing a data storage structure. Background technique

[0002] With the rapid development of big data and cloud computing, the amount of data in the world is increasing explosively. The existing distributed storage system can no longer meet the demand of data volume, and the traditional distributed storage has a large cluster, which leads to its equipment cost. And the operation and maintenance cost is expensive. In addition, the system framework of distributed storage needs the scheduling of the central node, which makes the user unable to write and read the data storage flexibly, efficiently and quickly. Therefore, a storage structure is proposed, which can Improve the shortcomings of existing distributed storage that cannot read and write quickly. Contents of the invention

[0003] The invention mainly solves the problem that distributed stora...

Examples

Embodiment 1

[0032] The present invention provides a method for optimizing data storage structure, please refer to figure 1 , including the following steps:

[0033] S100, use the kernel plug-in loading method to load openCAS into the ceph installation package, first install openCAS on the servers in the ceph storage cluster, introduce the driver module OCF into the openCAS plug-in, and OCF guides high-speed storage media, such as memory, optane Or SSD, etc., create the first response area through OCF, the first response area is equivalent to the cache process, but the first response area can respond faster than ordinary memory, the first response area supports write through and write back working modes, write through It is a direct write process, when receiving and updating data in the first response area, write data at the same time, and send the data to the back-end server and cache area, write back is a write-back process, when receiving data in the first response area , first write t...

Embodiment 2

[0038] The present invention also provides a system for optimizing data storage structure, which is applied to servers, please refer to figure 2 , including: the first response area, data fast copy cache area and common storage area;

[0039] The server is a ceph server, using the kernel plug-in loading method, loading openCAS into the ceph installation package, and creating the first response area;

[0040]The server accepts the read and write requests sent by the user. The first response area supports the work modes of write through and write back. The first response area supports the work modes of write through and write back. When updating data, write the data at the same time, and send the data to the back-end server and cache area, write back is the write-back process, when receiving data in the first response area, first write the data into the cache area, when the cache area When the data is replaced, the modified and replaced data will be sent to the backend server....

technical field

[0001] The invention relates to the field of data storage, in particular to a method, system and medium for optimizing a data storage structure. Background technique

[0002] With the rapid development of big data and cloud computing, the amount of data in the world is increasing explosively. The existing distributed storage system can no longer meet the demand of data volume, and the traditional distributed storage has a large cluster, which leads to its equipment cost. And the operation and maintenance cost is expensive. In addition, the system framework of distributed storage needs the scheduling of the central node, which makes the user unable to write and read the data storage flexibly, efficiently and quickly. Therefore, a storage structure is proposed, which can Improve the shortcomings of existing distributed storage that cannot read and write quickly. Contents of the invention

[0003] The invention mainly solves the problem that distributed stora...

Examples

Embodiment 1

[0032] The present invention provides a method for optimizing data storage structure, please refer to figure 1 , including the following steps:

[0033] S100, use the kernel plug-in loading method to load openCAS into the ceph installation package, first install openCAS on the servers in the ceph storage cluster, introduce the driver module OCF into the openCAS plug-in, and OCF guides high-speed storage media, such as memory, optane Or SSD, etc., create the first response area through OCF, the first response area is equivalent to the cache process, but the first response area can respond faster than ordinary memory, the first response area supports write through and write back working modes, write through It is a direct write process, when receiving and updating data in the first response area, write data at the same time, and send the data to the back-end server and cache area, write back is a write-back process, when receiving data in the first response area , first write t...

Embodiment 2

[0038] The present invention also provides a system for optimizing data storage structure, which is applied to servers, please refer to figure 2 , including: the first response area, data fast copy cache area and common storage area;

[0039] The server is a ceph server, using the kernel plug-in loading method, loading openCAS into the ceph installation package, and creating the first response area;

[0040]The server accepts the read and write requests sent by the user. The first response area supports the work modes of write through and write back. The first response area supports the work modes of write through and write back. When updating data, write the data at the same time, and send the data to the back-end server and cache area, write back is the write-back process, when receiving data in the first response area, first write the data into the cache area, when the cache area When the data is replaced, the modified and replaced data will be sent to the backend server....

Images