Method and system for collecting and storing stock data based on cloud computing technology

A data acquisition and cloud computing technology, applied in the field of network applications, can solve the problems of poor data structure flexibility, expensive software and hardware costs, and difficulty in horizontal expansion, and achieve easy expansion of data structure, strong real-time performance, and flexible relationship model structure Effect

Active Publication Date: 2013-02-06
武汉民大信息科技有限公司
View PDF2 Cites 18 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] General database systems cannot meet such large-scale, real-time, and flexible requirements
There are problems such as complex development, poor real-time performance, difficult management, and poor flexibility of data structure in the i

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for collecting and storing stock data based on cloud computing technology
  • Method and system for collecting and storing stock data based on cloud computing technology
  • Method and system for collecting and storing stock data based on cloud computing technology

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0037] Such as figure 1 As shown, the hardware application environment of the present invention is as follows: a cloud computing platform consisting of a client, a master server and multiple slave servers, and a remote stock data server. The stock data collection work request is submitted by the user to the main server through the client; the main server first decomposes the collection work into multiple collection Map tasks and assigns them to different slave servers for execution, and the main server is also responsible for metadata management related to data storage management ; The slave serve...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for collecting and storing stock data based on a cloud computing technology. The method comprises the following steps: obtaining a plurality of stock codes; generating a corresponding URL (uniform resource locator) list file for each stock code; setting fragment size of a Hadoop distributed file system of a cloud computing platform; storing the URL list file into the Hadoop distributed file system; receiving a stock data collecting request submitted by a user; resolving the stock data collecting request into corresponding quantity of Map tasks according to the quantity of the file fragments; assigning the resolved Map tasks to a slave server on which corresponding fragments are located to carry out; transmitting an Http (hyper text transport protocol) data request to a remote stock data server from a server according to the resolved Map tasks; returning the stock data by the remote stock data server; and analyzing the stock data according to the resolved Map tasks, wherein compiling of analysis function program abides by structure description of a remote stock server on the stock data. The method is simple to develop and good in data structure expansibility, and rapid collection of the stock data can be achieved.

Description

technical field [0001] The invention belongs to the field of network application technology, and more specifically relates to a method and system for collecting and storing stock data based on cloud computing technology. Background technique [0002] The characteristics of the stock data generated on the securities trading day are mainly real-time and large-scale. At present, each stock in Shanghai and Shenzhen stock markets will generate an update data every 5 seconds during the trading day, and the daily data growth capacity is between 1.5-2GB. According to relevant statistical data in 2011, the number of listed and traded stocks on the Shanghai and Shenzhen Stock Exchanges in China is about 1,800, and that on the Hong Kong Stock Exchange is about 6,700. In foreign countries, the number of stocks listed and traded on the NASDAQ stock market in the United States is about 5,800. If funds and bonds are taken into account, the amount of data generated at the same time is even...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08G06F17/30
Inventor 李成华江小平张华余良李凡
Owner 武汉民大信息科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap