Unlock instant, AI-driven research and patent intelligence for your innovation.

Spark-based big data development method and device, terminal

A technology of big data and development toolkit, which is applied in the direction of program code conversion, creation/generation of source code, software design, etc. It can solve the problems of insufficient packaging of development methods and inability to get started quickly, so as to improve development efficiency and speed up development progress , the effect of lowering the entry threshold

Active Publication Date: 2018-12-25
QIANXUN SPATIAL INTELLIGENCE INC
View PDF10 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The embodiment of the present invention provides a Spark-based big data development method, device, and terminal, aiming to solve the problem that the development method of the prior art is not fully packaged and cannot be used quickly

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Spark-based big data development method and device, terminal
  • Spark-based big data development method and device, terminal
  • Spark-based big data development method and device, terminal

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0044] figure 1 A flow chart of a Spark-based big data development method provided by the first embodiment of the present invention is shown, the method includes:

[0045] Step S1, installing an integrated development environment (Integrated Development Environment, IDE), to facilitate the introduction of template projects;

[0046] IDEs generally include code editors, compilers, debuggers, and graphical user interface tools. The IDE in the embodiment of the present invention may be a development environment such as IDEA and Eclipse. In step S1, after installing the integrated development environment, it is also necessary to install the Maven repository and the Maven plug-in of the IDE, so that the development tool can import the template project.

[0047] Step S2, downloading the latest template project, compiling and packaging at the same time, generating a software development kit (Software Development Kit, SDK);

[0048] The SDK in the embodiment of the present inventio...

Embodiment 2

[0071] Such as image 3 As shown, it is a structural diagram of a Spark-based big data development device provided by the second embodiment of the present invention. The device includes: an installation unit 1, a compilation unit 2 connected to the installation unit 1, and an add-on connected to the compilation unit 2. Unit 3. Development unit 4 connected to add unit 3, wherein:

[0072] Installation unit 1 is used to install the integrated development environment and facilitate the introduction of template projects;

[0073] IDEs generally include code editors, compilers, debuggers, and graphical user interface tools. The IDE in the embodiment of the present invention may be a development environment such as IDEA and Eclipse. In step S1, after installing the integrated development environment, it is also necessary to install the Maven repository and the Maven plug-in of the IDE, so that the development tool can import the template project.

[0074] The compilation unit 2 i...

Embodiment 3

[0092] Figure 4 A structural diagram of a terminal provided by the fourth embodiment of the present invention is shown, the terminal includes: a memory (memory) 41, a processor (processor) 42, and a bus 43, and the processor 42 and the memory 41 complete mutual communication via the bus 43 communication between them.

[0093] Memory 41, used for storing various data;

[0094] Specifically, the memory 41 is used to store various data, such as parameters and codes in the big data development process, which is not limited here, and the memory also includes multiple computer programs.

[0095] The processor 42 is used to call various computer programs in the memory 41 to execute a Spark-based big data development method provided by the first embodiment, for example:

[0096] Install the integrated development environment to facilitate the introduction of template projects;

[0097] Download the latest template project, compile and package at the same time, and generate a softw...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention is applicable to the technical field of big data development, and provides a Spark-based big data development method and device and a terminal. The big data development method comprisesthe following steps of: installing an integrated development environment to facilitate the introduction of template engineering; downloading the new template project, and compiling and packaging to generate the software development kit; adding the software development kit to the integrated development environment to form a unified development template; a large data development project being newlybuilt, and the development template being applied to carry out the large data development. In the invention, the template-based development mode not only provides the encapsulated class and method, but also provides the directly operable development template, which improves the development efficiency, reduces the entry threshold, and accelerates the development progress in the simplest and most effective manner.

Description

technical field [0001] The invention belongs to the technical field of big data development, and in particular relates to a Spark-based big data development method, device, and terminal. Background technique [0002] In recent years, more and more tool development kits have brought great convenience to our development tasks, that is, technicians use their own packaging methods to package some dependencies and practical methods, and then others use them by reference . This method is currently the most common technology and function sharing method, but this method also has certain disadvantages, that is, it is not friendly to beginners, and it is not thorough enough for Spark and other development packages, and it is difficult for many people to get started quickly. [0003] Existing tool development kits only provide encapsulation methods or parent classes, which are used through inheritance and reference. Users need to understand the internal methods to use them well, and n...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F8/33G06F8/41G06F8/20
CPCG06F8/20G06F8/33G06F8/41
Inventor 刘霄峰
Owner QIANXUN SPATIAL INTELLIGENCE INC