Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Massive data-oriented data exchange method

A technology for data exchange and massive data, applied in the field of information technology processing, it can solve problems such as reliability management, poor scalability, and complexity, and achieve the effect of reducing pressure and accelerating transmission speed.

Active Publication Date: 2014-03-26
HOHAI UNIV
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The biggest disadvantage of this method is poor scalability. When the number of clients that need to be exchanged increases, the sending tasks on the server side will double. When one client is added, the server needs to send 400G of data. When the number of clients is increased to 10, the data to be sent is 2000G. The pressure on the server is increasing. In addition, FTP commands and data are generally transmitted in plain text , the data security is low, and the ability to identify different customers is poor, so it is difficult to achieve different sending tasks for different users
The second is that IP multicast technology can effectively reduce the server and network load by its multiplexing method, but due to many deficiencies, such as the complexity of implementation, congestion control, reliability management, etc., it is difficult to practical application
However, the existing subscription / publishing system can only handle the first type of massive data. When the data is the second type of massive data, existing subscription / publishing systems such as Google Reader, Baidu Subscription, and GreatNews cannot process a single data in several Hundreds of megabytes or even several gigabytes of data are sent to subscribers

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Massive data-oriented data exchange method
  • Massive data-oriented data exchange method
  • Massive data-oriented data exchange method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] Below in conjunction with accompanying drawing, the technical scheme of invention is described in detail:

[0029] A data exchange system for massive data is mainly composed of data update control module, entity database, metadata registration module, metadata database, distribution strategy database, distribution strategy management module, subscription information matching / retrieval module, subscription management client server, Tracker The server consists of 9 parts, the structure diagram is as follows Figure 4 As shown, the basic functions of each part are introduced below.

[0030] The main functions of the data update control module: store data in the entity database; generate a torrent file with the extension .torrent according to the BT coding specification; pass the metadata information of the data to the metadata registration module; use the entity data as a super seed, And connect to the Tracker server.

[0031] The entity database is mainly responsible fo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a massive data-oriented data exchange method, which belongs to the technical field of information processing. The method comprises the following steps that: a data updating control module receives new data, and performs bit torrent (BT) coding on the data to form a torrent file; a distribution strategy management module performs searching to determine whether any user subscribes the data; if a certain user subscribes the data, the distribution strategy management module informs the subscribing user of downloading the torrent file, the subscribing user resolves the torrent file to obtain the address of a Tracker server, is connected with the Tracker server to obtain information about all Peers uploading or downloading the data, and uploads own information to the Tracker server, and all the Peers perform peer-to-peer (P2P) data exchange with one another; and if no user subscribes the data, the whole process is finished and the data exchange is not performed. By the method, the problems of low transmission speed, heavy server load, data updating and transmission untimeliness and the like in the transmission and exchange processes of massive data are solved.

Description

technical field [0001] The invention relates to a data exchange method for massive data, which belongs to the field of information technology processing. Background technique [0002] Massive data has emerged in recent years with the development of information technology, especially Internet technology, and is mainly used to describe huge and unprecedentedly vast data, such as various spatial data, report statistics, text, sound, images, hypertext and other environmental and cultural data information. Now, many enterprises and companies are involved in the operation and processing of massive data, such as water conservancy departments, meteorological departments, etc., handle very large amounts of data. To sum up, massive data mainly includes two types of data, one is that each piece of data itself is not large but the amount of data is huge, such as SMS service data in mobile communication; the other is that the data itself is very large, such as remote sensing images from...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08G06F17/30
Inventor 冯钧胥世民唐志贤付言章朱忠华杜龙刘子源顾忠国韦冕卞一路
Owner HOHAI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products