Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Distributed data processing method and system

A technology of distributed data and processing methods, applied in the field of data processing, can solve problems such as poor scalability, and achieve the effect of scalability and adaptability

Active Publication Date: 2015-03-25
新方正控股发展有限责任公司 +1
View PDF4 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] For this reason, the technical problem to be solved by the present invention is to overcome the problem of poor scalability of the entire distributed file system caused by the installation of plug-ins on the client in the prior art, and to provide a highly scalable client-side and server-side operation Distributed data processing method and system for

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed data processing method and system
  • Distributed data processing method and system
  • Distributed data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0087] As the distributed data processing method that a kind of server end operation of the present invention, such as figure 2 shown, including the following steps:

[0088] The server obtains resource information of the client in real time.

[0089] The server determines a task conforming to the processing capability of the client according to the resource information.

[0090] The server assigns and sends tasks that meet its processing capabilities to the client.

[0091] Wherein, the task includes the address of the data to be processed, the address of the plug-in library, and the processing instruction type required for processing the data to be processed. Plug-and-play plug-ins are stored in the plug-in library.

[0092] In the distributed data processing method run by the server of the present invention, the distributed task contains the address of the plug-in required to process the data to be processed, and the client can complete the data processing without insta...

Embodiment 2

[0094] As an embodiment of the present invention, on the basis of Embodiment 1, the distributed data processing method run on the server side further includes the following steps of updating the plug-in library:

[0095] Store new plugins into said plugin repository.

[0096] The processing instruction type corresponding to the new plug-in is stored in the server.

[0097] The version corresponding to the new plug-in is stored in the server. Correspondingly, the task may also include plug-in version information, and the specific form of the task including plug-in version information is as follows: .

[0098] During the normal operation of the server, if there is a new type of data processing, at this time, it is only necessary to update the plug-in library, and add the new type of data processing plug-in to the existing plug-in library. The task contains the address of the new plug-in, and when the client processes the task, it only needs to call the new plug-in according to...

Embodiment 3

[0100] As an embodiment of the present invention, on the basis of the above-mentioned embodiment 1 or embodiment 2, in the step of assigning and sending a task that meets the processing capability of the client to a client that meets the task processing conditions, the task It also includes the state of the data to be processed, and the state of the data to be processed further includes client information and tag information, wherein,

[0101] The client information is the information of the client assigned to the data to be processed; the marking information is information marking that the data to be processed has been taken away.

[0102]When the server assigns a new task to fetch data to be processed, all those with the tag information indicate that the data to be processed has been taken away and cannot be re-taken and distributed; it can effectively reduce or even prevent the same data to be processed from being distributed to The occurrence of resource waste caused by re...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a distributed data processing method and system. The method comprises the following steps that: a server allocates and sends a task which is consistent with the processing capacity of a client which satisfies a processing condition to the client; after receiving the task, the client processes the task to obtain a plug-in address and the address of data to be processed; the client loads a plug-in according to the plug-in address, and extracts the data to be processed according to the address of the data to be processed; and the client processes the data to be processed by using the plug-in to obtain processed data. When processing of a new type of data to be processed is added, the client processes the data to be processed in a way of calling the plug-in, so that only one plug-in for processing the new type of data to be processed needs to be developed. Moreover, the plug-in can be called by different clients by being put into a plug-in library without upgrading of the clients on a large scale, so that the method and the system have the advantage of high adaptability to processing of new types of data to be processed.

Description

technical field [0001] The present invention relates to a data processing method and system, in particular to a distributed data processing method and system run by a client and a server. Background technique [0002] The so-called distributed data processing is a computer data processing method that connects multiple computers in different locations or with different functions or with different data through a communication network, and completes information processing tasks in a coordinated manner under unified management and control. Distributed data processing is introduced in the splittable file data processing process, that is, the data processed by a single computer is processed concurrently and cooperatively by multiple computers, which can provide a strong technical guarantee for a large number of file data processing. When processing a large number of data files, especially when frequent image compression, file format conversion, etc. consume computer performance, t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04L29/08
Inventor 林厚超黄科
Owner 新方正控股发展有限责任公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com