Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Block task execution system for data synchronization based on MapReduce architecture

A data synchronization and task execution technology, applied in structured data retrieval, special data processing applications, electronic digital data processing, etc., to achieve the effect of improving efficiency and reducing burden

Pending Publication Date: 2021-11-05
中建材信息技术股份有限公司 +1
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, there is currently no relatively complete block task execution system for data synchronization based on the MapRedce architecture

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Block task execution system for data synchronization based on MapReduce architecture
  • Block task execution system for data synchronization based on MapReduce architecture
  • Block task execution system for data synchronization based on MapReduce architecture

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0064] Such as Figure 1-Figure 8 As shown, this embodiment provides a block task execution system for data synchronization based on MapReduce architecture, including

[0065] Infrastructure unit 100, data processing unit 200, data synchronization unit 300 and task parallel unit 400; infrastructure unit 100, data processing unit 200, data synchronization unit 300 and task parallel unit 400 are sequentially connected through network communication; infrastructure unit 100 uses It is used to build and manage the programming framework of the distributed computing program that supports the operation of the system and to perform task scheduling and management operations on this basis; the data processing unit 200 is used to complete the function calculation process of data based on the MapReduce programming steps; the data synchronization unit 300 It is used to manage and select the most suitable method to complete data synchronization; the task parallel unit 400 is used to manage a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of data synchronization, in particular to a block task execution system for data synchronization based on a MapReduce architecture. The system comprises an infrastructure unit, a data processing unit, a data synchronization unit and a task parallel unit; the infrastructure unit is used for building and managing a programming framework of a distributed operation program supporting system operation; the data processing unit is used for completing a function calculation processing process of the data; the data synchronization unit is used for completing data synchronization; and the task parallel unit is used for realizing a parallel processing process of a plurality of tasks. According to the design, mass and large-scale data can be rapidly analyzed and processed, and the data processing efficiency is improved; parallel computing of distributed storage, data communication, fault-tolerant processing and the like of data can be automatically carried out, and the burden of software developers is reduced; and a data synchronization method and a task parallel method can be identified and called, parallel execution of tasks among different programs is realized, and the degree of parallelism among the tasks of the different programs can be evaluated.

Description

technical field [0001] The invention relates to the technical field of data synchronization, in particular to a block task execution system for data synchronization based on MapReduce architecture. Background technique [0002] Regarding data synchronization, it mainly includes two levels of synchronization. One is to realize data synchronization through background program coding, and the other is to directly act on the database to realize data synchronization at the database level. Among them, there are three main methods of database synchronization at the database level: synchronization through publish / subscribe, data synchronization through SQL JOB, and data synchronization through Service Broker message queues. In the process of massive data analysis and processing, one, two or more data synchronization methods can be carried out simultaneously, and each method corresponds to one or more tasks. When these tasks are carried out simultaneously, execution confusion and data...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/48G06F9/50G06F9/54G06F16/182G06F16/27
Inventor 王伟王洋刘庆冀晓镭
Owner 中建材信息技术股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com