Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Shapefile master-slave type parallel writing method based on OGR (open geospatial rule)

A master-slave and file technology, applied in the direction of multi-programming devices, etc., can solve problems such as no consideration, achieve the effect of improving overall efficiency and solving data coverage problems

Inactive Publication Date: 2012-07-18
NANJING UNIV
View PDF3 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Previous studies mostly focused on the parallelization of data processing methods, without considering that when the amount of data is very large, the I / O (input / output) time between data and disk will seriously affect the efficiency of program execution, and among them, vector data A large amount of geographic data is mostly stored in shapefile files, so the invention of the parallel writing method for shapefile files is of great significance to the efficient processing of shapefile files

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Shapefile master-slave type parallel writing method based on OGR (open geospatial rule)
  • Shapefile master-slave type parallel writing method based on OGR (open geospatial rule)
  • Shapefile master-slave type parallel writing method based on OGR (open geospatial rule)

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] In this embodiment, the attached image 3Changsha's land use status survey map (changsha.shp) is shown as the test data written in master-slave parallel mode based on the OGR shapefile. The number of spots in the test data is 692176, the data size is 932M, and the total area covered is 11819.46 square kilometers.

[0027] The specific operating environment of this example is the IBM System x3500-M3X series server environment. The server hardware configuration is 2 CPUs, (the specification is Intel Xeon Quad Core E5620, the main frequency is 2.4GHz, 12MB Cache, quad-core); the memory is 8GB (two 4GB memory sticks, the specification is DDR3 1333MHz LP RDIMM); the hard disk is 2TB (4 500GB hard disks, specifications are 7.2K 6Gbps NL SAS 2.5-inch SFF Slim-HS HD), and the network is an integrated dual-port Gigabit Ethernet. Software configuration: Centos Linux is selected as the operating system, OpenMPI is selected as the implementation product of MPI, version 1.4.1 or a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the field of high-performance geographic calculation, and discloses a shapefile master-slave type parallel writing method based on the OGR (open geospatial rule), which includes: step 1, inputting command line parameters; step 2, setting up a shapefile target data source and a layer file serially, and closing a target data source file; step 3, opening a shapefile original file to be processed to acquire each layer and graphic data volumes of the shapefile; step 4, performing MPI (message passing interface) parallel initialization to acquire numbers and quantity of processes, and setting a master process and slave processes; step 5, performing data partitioning to determine initial fid and terminal fid of graphics in the shafpefile processed by each process; and step 6, allowing each slave process to perform shapefile data processing. Computing resources are fully used, and integral efficiency of shapefile processing is improved.

Description

technical field [0001] The invention belongs to the field of high-performance geographic computing, and in particular relates to an OGR-based master-slave parallel writing method for shapefiles. Background technique [0002] With the rapid progress of earth observation technology, massive multi-dimensional spatio-temporal data is increasing day by day, and geographical information is becoming more and more abundant. Geographic vector data written to disk after processing is becoming more and more dense. A format for non-topological entity vector data structures of geometric and attribute features) files are the most commonly used vector data files. The serial writing mode and traditional hardware platform of the existing geographic information system have been difficult to support the demand for fast writing of massive shapefiles. With the gradual popularization of new hardware architectures of parallel computing clusters and multi-core processors, it provides an opportunit...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/46
Inventor 蒲英霞靳志宾陈振杰李飞雪陈冲姜晓慧李满春黄秋昊
Owner NANJING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com