Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Analysis task processing method and device based on Flink technology and storage medium

A technology for analyzing tasks and processing methods, applied to devices and storage media, and in the field of analysis task processing methods based on Flink technology, which can solve problems such as the inability to automatically process analysis task requests from external big data platforms

Active Publication Date: 2020-05-15
WUHAN CITMS TECH CO LTD
View PDF4 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The embodiment of the present application provides an analysis task processing method, device and storage medium based on Flink technology. In the prior art, the analysis task request of the external big data platform cannot be automatically processed, and the analysis task request for the external big data platform can only be done manually. The case of splicing complex command parameters simplifies the operation complexity in the big data analysis scenario, automatically splicing shell commands for analysis tasks, and the operation is simple and easy, which reduces the requirements for operators of analysis tasks and does not require operators to carry out complex tasks. work, to avoid errors in the process of human operation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Analysis task processing method and device based on Flink technology and storage medium
  • Analysis task processing method and device based on Flink technology and storage medium
  • Analysis task processing method and device based on Flink technology and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0075] In describing the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", " The orientation or positional relationship indicated by "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation shown in the drawings Or positional relationship is only for the convenience of describing the present i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention discloses an analysis task processing method and device based on the Flink technology and a storage medium. According to the analysis task processing method based on the Flink technology, when an analysis task is executed; the method comprises the steps of obtaining a task logic data packet of a target analysis task; searching parameters and a command template configured for the target analysis task in advance; automatically splicing to obtain a complete shell command for executing a target analysis task; completing the execution of a target analysis task through a shell command. According to the embodiment of the invention, an analysis task request of an external big data platform cannot be automatically processed in the prior art; aiming at the condition that an analysis task request of an external big data platform can only be manually spliced with complex command parameters, the operation complexity in a big data analysis scene is simplified, the shell command of the analysis task is automatically spliced, the operation is simple and easy, the requirements on an analysis task operator are reduced, the operator does not need to carry out complex work, and errors occurring in the manual operation process are avoided.

Description

technical field [0001] The present invention relates to the technical field of big data processing, in particular to a Flink technology-based analysis task processing method, device and storage medium. Background technique [0002] There are many technologies for big data computing and processing, such as Hadoop (batch processing), Storm (stream processing), Samza (stream processing), Flink (batch stream processing) and other technologies, no matter which technology is used for big data analysis , need to allocate space and resources for processing tasks, and control and manage all jobs. [0003] At present, these big data processing technical frameworks have their own job management functions, and other application frameworks provide single-function job management and resource allocation functions, which are more powerful and professional, such as the integrated mode of Flink and yarn. In the integrated mode of Flink and yarn, for the operation of each task job, you first ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/4881G06F9/5027
Inventor 詹聪唐志斌蔡青
Owner WUHAN CITMS TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com