Distributed stream-oriented computation based spatial data processing method and system

A technology of spatial data and streaming computing, which is applied in digital data processing, special data processing applications, calculations, etc., can solve problems such as inability to process massive geospatial data in real time, and achieve real-time and strong real-time effects

Inactive Publication Date: 2016-04-20
CHINESE ACAD OF SURVEYING & MAPPING
View PDF6 Cites 22 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Aiming at the problem that real-time processing of massive geospatial data cannot be realized in the prior art,

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed stream-oriented computation based spatial data processing method and system
  • Distributed stream-oriented computation based spatial data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] As a preferred specific implementation of the present invention, a spatial data processing method and system based on distributed streaming computing is provided.

[0027] see figure 1 and figure 2 , the spatial data processing method based on distributed streaming computing of the present invention comprises the following steps:

[0028] Create a Kafka scheduled task to extract geospatial data from the data source after a certain period of time set by the Kafka scheduled task. Kafka is a high-throughput distributed message publishing and subscription system, which can realize the transmission of streaming data.

[0029] Create a message queue for the geospatial data to flow in, and classify the message queue according to the preset spatial data type, thereby specifying the tags in the message queue; according to the complexity and diversity of the geospatial data, Classifying geospatial data mainly includes: digital line drawing data, image data, digital elevation ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a distributed stream-oriented computation based spatial data processing method and system. A message queue of Kafka is established and used for real-time monitor, so that when data are stored in the message queue, Kafka can timely draw the monitored data, further, the data are processed in real time, and the real-time performance of data processing is realized. Besides, data are stored in a database and are further backed up, and the processed data can be updated to the database after the data are processed, so that the data in the database are updated. Real-time computation of geospatial big data is realized through distributed stream-oriented computation, and the method and system can support development and application of geospatial data with high real-time performance.

Description

[technical field] [0001] The invention relates to the fields of geographical information and computers, in particular to a processing method for geographical space information based on distributed stream computing. [Background technique] [0002] With the development of modern surveying and mapping equipment technologies such as mobile hyperspectral remote sensing, synthetic aperture radar, and 3D laser scanning, and the widespread use of wireless positioning equipment such as mobile phones, personal digital assistants, and vehicle navigation, more and more spatial data are generated. The real-time processing of data becomes a huge challenge. [0003] At present, multi-core parallel computing is mainly used to process massive spatial data, and similar Hadoop clusters can also be used to process massive spatial data. However, limited to the computing power of a single computer and the weakness of the Hadoop cluster's poor real-time data processing using the MapReduce method,...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/254
Inventor 刘纪平仇阿根何旺君张用川
Owner CHINESE ACAD OF SURVEYING & MAPPING
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap