Unlock instant, AI-driven research and patent intelligence for your innovation.

A spatial data caching method, device and storage medium

A technology of spatial data and data, applied in the field of big data storage, can solve the problems of lower system efficiency and lower cache hit rate, etc.

Active Publication Date: 2020-12-18
MIGU CO LTD +1
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

If only Hadoop's own caching mechanism is used, when the user accesses the adjacent area, if the adjacent area is not in the memory, it needs to read the corresponding data from the disk, which increases the disk I / O and may reduce the cache capacity. hit rate, resulting in reduced system efficiency

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A spatial data caching method, device and storage medium
  • A spatial data caching method, device and storage medium
  • A spatial data caching method, device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0092] In various embodiments of the present invention, an operation request is received; the operation request is used to request to read a data block or to request to write a data block; determine the data block corresponding to the operation request, and add the data block to the cache Write into the queue; determine the priority of the data block, and according to the priority of the data block, when determining that the data block obtains the permission to join the cache queue, add the data block to the cache queue; the priority and The degree of association between the data block and the reference cache block in the cache queue is related.

[0093] The present invention will be described in further detail below in conjunction with the examples.

[0094] figure 1 A schematic flowchart of a spatial data caching method provided by an embodiment of the present invention; the method can be applied to data nodes, such as figure 1 As shown, the method includes:

[0095] Step...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a spatial data caching method, which comprises the following steps of receiving an operation request, wherein the operation request is used for requesting to read the data block or requesting to write the data block, determining a data block corresponding to the operation request, adding the data block into a buffer pre-writing queue; determining the priority of the data block, and adding the data block to the cache queue when the data block obtains the right to join the cache queue according to the priority of the data block, wherein the priority is related to a degreeof association of the data block and a reference buffer block in the buffer queue. The invention also discloses a spatial data buffer device and a computer-readable storage medium.

Description

technical field [0001] The invention relates to big data storage technology, in particular to a spatial data caching method, device and computer-readable storage medium. Background technique [0002] With the maturity of big data processing technology, Hadoop has become a popular tool for big data processing. Hadoop disperses and stores data in a DataNode cluster. When running a MapReduce task, it decides which machine to assign the task to according to the distribution of the data to achieve efficient distributed computing. Hadoop itself is an input / output (I / O, Input / Output) intensive data processing framework, I / O efficiency is crucial to its performance, and caching technology can effectively reduce the number of I / O operations and improve I / O / O performance to optimize the performance of the Hadoop cluster. [0003] Hadoop itself provides a caching mechanism that allows users to configure the files to be cached. When a file is set as the file to be cached, the DataNod...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F12/0895G06F16/172
CPCG06F12/0895G06F2212/1021
Inventor 李宗祥严国友孙一波
Owner MIGU CO LTD