Online analysis image and grid cube data storage management method and system
A technology for data storage and management methods, applied in the field of big data, which can solve the problems of high uncertainty and low performance
Active Publication Date: 2021-05-18
FUZHOU UNIV
View PDF5 Cites 1 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0003] The traditional MongoDB image storage method is to put the metadata of remote sensing image data into files and store them in the fs.files of GridFS, fs.chunks store the binary files of the image, and then specify the slice key for fragmentation to support distributed storage and It supports horizontal expansion of the database and is suitable for managing large-scale remote sensing image data. When performing cube online analysis, it is necessary to retrieve relevant 2D images first, and then perform cutting and reconstruction after obtaining all required images. If relevant images are missing or have resolution, etc. The parameters are inconsistent, and intervention in the refactoring process is required, the uncertainty is large, and the performance is not high
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0014] The technical solution of the present invention will be specifically described below in conjunction with the accompanying drawings.
[0015] like figure 1 As shown, an embodiment of the present invention provides an online instant analysis image and grid cube data storage method, the main processing steps include: storing the image and grid cube in various tile segmentation schemes in the online instant analysis image With MongoDB on the grid cube application service platform, each storage scheme corresponds to a configuration, and the index range of the cube request in the online analysis application is obtained, and the I / O number and data size of each configuration are compared through matching rules to obtain data volume conditions. The following tile partition storage scheme with the least I / O, access the target data block of the corresponding storage partition scheme, and analyze the application after completion.
[0016] Store images and grid cubes in GridFS wit...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to an online analysis image and grid cube data storage management method and system. The method comprises the steps that an image and a grid cube are stored in various tile segmentation schemes, each storage scheme corresponds to one configuration, the I / O number and the data size of each configuration are compared through a matching rule, the tile segmentation storage scheme with the minimum I / O under the condition that the data size is considered is obtained, a target data block of the corresponding tile segmentation storage scheme is accessed, and the target data block of the corresponding tile segmentation storage scheme is stored. Therefore, the online analysis performance of the cube is improved. According to the MongoDB-based cube data storage method provided by the invention, the access efficiency of the cube target data block under the online analysis application condition is improved, the transmission of invalid data blocks and the consumption of corresponding computing power are reduced, and the online analysis application performance is improved.
Description
technical field [0001] The invention belongs to the technical field of big data, and in particular relates to a method and system for online instant analysis of image and grid cube data storage and management. Background technique [0002] Today has entered the era of big data, especially the continuous development of earth observation applications, which puts forward new requirements for online analysis applications of multi-dimensional grids. The data volume of multi-dimensional grids increases geometrically with the increase of dimensions, and online analysis applications often Only part of the data is needed, and the full data access method of traditional images and grids wastes bandwidth and cannot meet the performance requirements of online analysis applications. With the continuous development and maturity of NoSQL database, it can better solve the application requirements of mass storage and online analysis. An important feature of the NoSQL database MongoDB is that...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/061G06F3/064Y02D10/00
Inventor 余劲松弟
Owner FUZHOU UNIV