Unlock instant, AI-driven research and patent intelligence for your innovation.

Distributed storage data reading efficiency optimization method, system, equipment and medium

A distributed storage and data reading technology, which is applied in the database field, can solve the problems of high impact on upper-level business and slow IO response rate, etc., and achieve the effect of increasing storage space and improving data reading efficiency

Pending Publication Date: 2021-12-10
北京东方金信科技股份有限公司
View PDF0 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] If the copy data is written directly to HDD instead of SSD, the entire space of SSD can be used to store the master data, but the upper-layer random read IO, random write IO (a large number of copies are directly written to the random IO of hdd), and write back to HDD The three IOs will compete for resources, resulting in an overall slowdown of the IO response rate, which has a relatively large impact on the upper-layer business

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed storage data reading efficiency optimization method, system, equipment and medium
  • Distributed storage data reading efficiency optimization method, system, equipment and medium
  • Distributed storage data reading efficiency optimization method, system, equipment and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0055] Such as figure 2 As shown, a distributed storage data reading efficiency optimization method provided in this embodiment includes the following steps:

[0056]1) Initialize the SSD, and divide the SSD space into two partitions, the primary partition and the elimination partition, which are used to store the primary data and copy data respectively.

[0057] It should be noted that, in the embodiment of the present invention, the primary partition of the SSD space is used to store primary data, and the eliminated partition is used to store copy data.

[0058] Specifically, when the embodiment of the present invention initializes the SSD, it mainly includes setting the ratio of the eliminated partition and the ratio of the flushing water level of the eliminated partition.

[0059] Such as image 3 As shown, the proportion of eliminated partitions refers to the percentage of eliminated partitions to the total capacity of SSD space.

[0060] Optionally, for an SSD with a...

Embodiment 2

[0100] Embodiment 1 above provides a method for optimizing read efficiency of distributed storage data. Correspondingly, this embodiment provides a system for optimizing read efficiency of distributed storage data. The system provided in this embodiment can implement the distributed storage data reading efficiency optimization method in Embodiment 1, and the system can be implemented by software, hardware, or a combination of software and hardware. For example, the system may include integrated or separate functional modules or functional units to execute corresponding steps in the methods of Embodiment 1. Since the identification system of this embodiment is basically similar to the method embodiment, the description process of this embodiment is relatively simple. For relevant parts, please refer to the part of the description of Embodiment 1. The embodiment of the system of this embodiment is only illustrative .

[0101] The distributed storage data reading efficiency opti...

Embodiment 3

[0106] This embodiment provides a processing device corresponding to the distributed storage data reading efficiency optimization method provided in Embodiment 1. The processing device may be a processing device for a client, such as a mobile phone, a notebook computer, a tablet computer, or a desktop computer. Computer etc., to carry out the method of embodiment 1.

[0107] The processing device includes a processor, a memory, a communication interface and a bus, and the processor, the memory and the communication interface are connected through the bus to complete mutual communication. A computer program that can run on the processor is stored in the memory, and the processor executes the method for optimizing the reading efficiency of distributed storage data provided in Embodiment 1 when running the computer program.

[0108] In some implementations, the memory may be a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a distributed storage data reading efficiency optimization method, a system, equipment and a medium, and the method comprises the steps: carrying out initialization of an SSD, dividing the space of the SSD into a main partition and an elimination partition, which are respectively used for storing main data and copy data; receiving an upper layer IO request, and reading and writing IO data according to the received IO request; and after a preset triggering condition is met, writing back the data stored in the SSD elimination partition to the HDD. According to the method, the SSD is divided into two logic areas, namely the main partition and the elimination partition, so that the storage space of the master data in the SSD can be effectively increased, and more master data can be stored. Meanwhile, the proportion of the dirty data amount stored in the SSD elimination partition is reasonably set, the HDD can write back the dirty data amount stored in the SSD elimination partition in time, and the data reading efficiency is improved. The method can be widely applied to the field of data reading.

Description

technical field [0001] The invention relates to a distributed storage data reading efficiency optimization method, system, equipment and medium, belonging to the technical field of databases. Background technique [0002] In the application where the database realizes the separation of computing and storage, the data is stored in the distributed storage cluster, and the data read and write efficiency of the database is directly affected by the read and write efficiency of the storage layer. In order to improve the read and write efficiency of the storage layer, the current distributed storage In an application, a medium such as an SSD (Solid State Disk or Solid State Drive, solid state disk) is usually used as a cache of a HDD (Hard Disk Drive, disk drive), so as to play an acceleration role. In order to ensure the reliability of stored data, the industry basically uses multiple copies to store the master and copy of IO data on different nodes; at the same time, for IO data,...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F16/22
CPCG06F3/0608G06F3/061G06F3/0631G06F3/0644G06F3/067G06F16/2246
Inventor 储飞王伟哲贺岩张海亮王松楠
Owner 北京东方金信科技股份有限公司