Intelligent data service method based on distributed system

A technology of distributed systems and data services, applied in the field of intelligent data services based on distributed systems, can solve the problems of difficult data services, difficult to form intelligent data services, adding preprocessing, etc., to achieve the effect of reducing bandwidth requirements

Active Publication Date: 2015-02-11
JIANGNAN INST OF COMPUTING TECH
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Non-relational databases provide Key-Value (key-value) storage, but it is difficult to add preprocessing functions to data services and form intelligent data services

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Intelligent data service method based on distributed system
  • Intelligent data service method based on distributed system
  • Intelligent data service method based on distributed system

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0027] The present invention is an intelligent data service method for data-intensive applications derived from a traditional distributed system.

[0028] specifically, figure 1 It schematically shows the architecture of the intelligent data service platform based on the distributed system according to the first embodiment of the present invention.

[0029] like figure 1 As shown, similar to most distributed file systems and distributed databases, the entire architecture is a typical master-slave (Master / Slave) architecture. E.g, figure 1 The shown distributed system-based intelligent data service platform architecture includes a master node M and multiple slave nodes; specifically, figure 1 shows the case of including n slave nodes, that is, the first slave node S1, the second slave node S2, the third slave node S3, . . . , the nth slave node Sn.

[0030] Wherein, the master node M includes a data preprocessing analysis engine M1 and a global metadata management module M2...

no. 2 example

[0038] figure 2A schematic diagram of a process of writing a file to a distributed system according to the second embodiment of the present invention is schematically shown. Among them, the process of writing files to the distributed system does not preprocess the data. Here, it is assumed that there are three slave nodes in the distributed system: the first slave node S1, the second slave node S2, and the third slave node S3, but obviously the number of slave nodes in the distributed system is not limited to three, but can be is any suitable number.

[0039] Specifically, as figure 2 As shown, the process of writing files to the distributed system according to the second embodiment of the present invention includes:

[0040] The first writing step a1: the client pcm1 asks the master node M whether the file to be written exists in the distributed system.

[0041] The second writing step b1: If the file to be written exists in the distributed system, the master node M sen...

no. 3 example

[0046] image 3 A schematic diagram of a process of reading a file from a distributed system according to the third embodiment of the present invention is schematically shown. Here, it is assumed that there are three slave nodes in the distributed system: the first slave node S1, the second slave node S2, and the third slave node S3, but obviously the number of slave nodes in the distributed system is not limited to three, but can be is any suitable number.

[0047] The process of reading a file from a distributed system according to the third embodiment of the present invention includes:

[0048] The first reading step a2: the client pcm1 sends a data request to the master node M, which includes the file path and required preprocessing.

[0049] The second reading step b2: the master node M analyzes the data request of the client pcm1, can determine the slave node where the required file is located and the required preprocessing program, directly performs the preprocessing ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an intelligent data service method based on a distributed system. A master node of the distributed system is used for managing global file namespace; and in processes of writing files into the distributed system and reading files from the distributed system, the master node of the distributed system is used for analyzing and processing requests of clients, selecting specific data preprocessing programs, distributing the programs to slave nodes of the distributed system for subsequent data preprocessing and transmission. According to the intelligent data service method based on the distributed system, prior distributed system storage space can be clustered rapidly in data intensive application environments; computing resources of the distributed system are fully used, so that data services can be provided intelligently according to requests of external computing devices; and part of data processing loads are further transferred from external computing devices to the distributed system, and accordingly, bandwidth requirements for providing data services for external computing devices are reduced.

Description

technical field [0001] The present invention relates to the technical field of computing, and more specifically, the present invention relates to an intelligent data service method based on a distributed system. Background technique [0002] In data-intensive applications, large-scale data set processing is the core of the application, and I / O (input / output) bandwidth becomes the main factor affecting its performance, which makes traditional computing and storage separation systems unsuitable for data-intensive applications. Applications, therefore, require a new type of data storage and service model to improve the performance of data transmission and processing. [0003] Data service generally refers to the storage, management and transmission of data, and its specific manifestations are different in different application scenarios. In the context of data-intensive applications, data services mainly include two aspects, one is the storage technology of massive data, and t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 谢向辉臧春峰吴东郝子宇原昊钱磊张鲁飞胡苏太
Owner JIANGNAN INST OF COMPUTING TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap