Distributed processing method and device for spatial big data

A distributed processing and spatial data technology, applied in the field of data processing, can solve the problems of increased calculation time, time-consuming, and inefficient processing of massive spatial data, and achieve the effect of reducing time-consuming and improving processing efficiency

Pending Publication Date: 2022-02-25
BEIJING SUPERMAP SOFTWARE CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Due to the limitations of the IT technical framework and the technical requirements of big data on distributed storage and computing, the existing data analysis and calculation methods cannot handle massive spatial data well. Indexing takes a certain amount of time, which leads to an increase in the overall calculation time, and the processing of massive spatial data is not efficient enough

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed processing method and device for spatial big data
  • Distributed processing method and device for spatial big data
  • Distributed processing method and device for spatial big data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0042] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0043] As used herein, the term "including" and variations thereof are open-ended inclusions, ie, "including but not limited to". The term "based on" is "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the de...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a distributed processing method and device for spatial big data, and the method comprises the steps: obtaining spatial data which comprises vector data; according to a preset spatial index mode matched with the data distribution characteristics of the spatial data, conducting grid geographic partitioning on the spatial data, generating a spatial index file, wherein the spatial index file at least comprises a plurality of element object files; storing a plurality of element object files in the spatial index file to a distributed file system HDFS, so as to enable each computing node to obtain a plurality of target element object files with the same element object file name from each storage node of the distributed file system HDFS by calling the computing nodes in the Spark in parallel when spatial data computing is needed, and carrying out data processing by utilizing the spatial data contained in the plurality of target element object files. According to the invention, the processing efficiency of the spatial data can be improved.

Description

technical field [0001] The invention relates to the technical field of data processing, and more particularly, to a distributed processing method and equipment for spatial big data. Background technique [0002] With the increasing amount of spatial data obtained by traditional surveying and mapping methods, new surveying and mapping methods, and the Internet, and these massive spatial data also have different data structures and storage methods, how to perform high-performance on massive spatial data? Distributed storage, management, and analytical computing for the [0003] Due to the limitations of IT technology framework and the technical requirements of big data for distributed storage and computing, the existing data analysis and calculation methods cannot handle massive spatial data well. Indexing takes a certain amount of time, which increases the overall computing time and makes the processing of massive spatial data inefficient. SUMMARY OF THE INVENTION [0004...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/2453G06F16/29G06F16/13G06F16/172G06F16/182G06F9/50
CPCG06F16/2272G06F16/2246G06F16/24532G06F16/29G06F16/134G06F16/172G06F16/182G06F9/5027
Inventor 宋关福卢浩范善策郑美玲刘芳左健王柳张念娟周孙宇唐育涵
Owner BEIJING SUPERMAP SOFTWARE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap