Unlock instant, AI-driven research and patent intelligence for your innovation.

Method for deploying Spark application

A paragraph and plug-in technology, applied in the field of data processing, can solve problems such as time-consuming, inability to provide code prompts, verification, change data statistics latitude, etc., to achieve the effect of improving writing efficiency

Pending Publication Date: 2019-11-29
CHENDU PINGUO TECH CO LTD
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] With the gradual expansion of the business, the dimension and magnitude of the data and the timeliness of the operator's demand for data reports are constantly increasing and changing, but the existing data analysis business cannot support changing the latitude of data statistics anytime and anywhere, and quickly Generate data report
Usually it is to write a new set of business logic codes to adapt to the current data report requirements, which takes a long time and cannot provide verification data reports faster
[0004] In addition, since the Zeppelin tool is a web-based notebook tool, it cannot provide functions such as code hints and verification similar to those in the IDE, and it has a certain degree of complexity in writing code

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for deploying Spark application
  • Method for deploying Spark application
  • Method for deploying Spark application

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0012] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0013] The present invention provides a Zeppelin plug-in based on IDE (such as Intellij Idea), and the plug-in can perform the following steps:

[0014] 1. Set the Web access address of the Zeppelin tool, enter the name of the Notebook in the IDE, select the name, and use the shortcut key to create a new notebook for subsequent operations in the Zeppelin tool.

[0015] 2. Write business logic code and add or update paragraph information in the notebook. There are two situations:

[0016] (1) When the current code has only one line, place the cursor at the end of the code, and synchronize the code content of the line where the cursor is located to the notebook created in the previous step through the shortcut key. If the current paragraph does not exist, a new p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for deploying a Spark application. The method comprises the following steps: creating a Zeppelin Notebook by utilizing a plug-in; newly adding or updating predetermined paragraph information; running the paragraph information and checking an output result; and performing visual processing on the data in the notebook computer in a Web page of the Zeppelin tool, andrunning the whole notebook computer or a data result. According to the technical scheme provided by the invention, the data report can be efficiently generated, so that quicker and more reliable dataguarantee is provided for visual data requirements of operators.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a method for deploying Spark applications. Background technique [0002] Zeppelin is an Apache incubation project. It is an interactive data query and analysis tool in the form of Web notes. It can use Scala and SQL to query and analyze data online and generate reports. The background data engine of Zeppelin is Spark, and other data engines implemented by developers can also be added. The present invention is only aimed at the Spark data processing engine. The role of the execution engine is to execute the code entered in the notebook, such as figure 2 shown. [0003] With the gradual expansion of business, the dimension and magnitude of data and the timeliness of operators' demand for data reports are constantly increasing and changing, but the existing data analysis business cannot support changing the latitude of data statistics anytime, anywhere, quickly Generate d...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F8/30G06F8/60
CPCG06F8/30G06F8/60
Inventor 姚军徐滢
Owner CHENDU PINGUO TECH CO LTD