Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Distributed heterogeneous processing framework construction method and system

A construction method and framework system technology, applied in the field of heterogeneous processing framework construction, can solve problems such as difficult to meet the requirements of the underlying architecture, bloated vertical application architecture, difficult to maintain, etc., and achieve the effect of improving real-time processing efficiency and

Active Publication Date: 2020-01-03
CHENGDU UNIV OF INFORMATION TECH
View PDF8 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] With the continuous development and growth of the Internet industry, the amount of data to be processed and stored by the website every day increases exponentially, and the scale of the website is also expanding. The traditional vertical application architecture is becoming more and more bloated, difficult to maintain, high latency, and low throughput. It is difficult to meet the underlying architecture requirements of the big data information age. Using distributed architecture for data storage and computing is an inevitable trend in the development of the Internet era. At the same time, it is necessary to have efficient real-time processing efficiency on data lines and message storage capabilities. architecture to fill today's market technology needs

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed heterogeneous processing framework construction method and system
  • Distributed heterogeneous processing framework construction method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, the specific implementation manners of the present invention will now be described with reference to the accompanying drawings.

[0040] Such as figure 1 As shown, the construction method of the distributed-based heterogeneous processing framework depends on the distributed-based heterogeneous processing framework system, and can also be implemented independently, including the following steps:

[0041] S1. Start the Netty server in the Linux environment, listen to the specified port, and wait for the Socket connection;

[0042] The web application sends data to the Netty server;

[0043] Send the request data to the service scheduling module by the publishing module, the request data includes the first message, as follows:

[0044] S11. Format the data to be imported including the first message into a string form of json and encapsulate it as request dat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a distributed heterogeneous processing framework construction method. The method comprises the following steps that a publishing module sends request data and blocks a currentthread by subscribing a first message; a service scheduling module receives the request data, encapsulates the request data into an intermediate file package, writes the intermediate file package intothe cache module and blocks the current thread by subscribing a second message; a calculation engine module calls the intermediate file package, loads and processes corresponding data according to the configuration data, and stores the processed final file package to the cache module; the calculation engine module issues a second message to the service scheduling module; the service scheduling module receives the second message, identifies a second end mark and cancels subscription of the second message; the service scheduling module issues a first message to the issuing module; the publishing module receives the first message, identifies a first end mark and cancels subscription of the first message; the publishing module reads the final file package from the cache module to obtain a data processing result. According to the invention, the real-time processing efficiency on the data line and the message storage capability are improved.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a method and system for constructing a distributed-based heterogeneous processing framework. Background technique [0002] With the continuous development and growth of the Internet industry, the amount of data to be processed and stored by the website every day increases exponentially, and the scale of the website is also expanding. The traditional vertical application architecture is becoming more and more bloated, difficult to maintain, high latency, and low throughput. It is difficult to meet the underlying architecture requirements of the big data information age. Using distributed architecture for data storage and computing is an inevitable trend in the development of the Internet era. At the same time, it is necessary to have efficient real-time processing efficiency on data lines and message storage capabilities. The architecture to fill the technical need...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/27G06F9/48
CPCG06F9/4881
Inventor 曹亮陈冲
Owner CHENGDU UNIV OF INFORMATION TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products