Method, apparatus, electronic device, computer readable storage medium for simplifying programming

A computer and grammar technology, applied in the field of data processing, can solve problems such as increased learning costs, high learning costs, and unfavorable use by ordinary users, and achieve the effects of improving development efficiency, reducing continuous development, and simplifying programming tasks

Active Publication Date: 2019-01-01
深圳市木瓜移动科技有限公司
View PDF6 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The big data (Apache Spark) platform has the characteristics of high flexibility and strong programmability, but it also increases the learning cost for our ordinary programming and operation and maintenance personnel while being flexible.
Mainly in daily business, if you need to add new statistical tasks, you need to rewrite the interface to complete code editing, compiling and debugging. On the one hand, relatively speaking, a lot of repetitive and unnecessary work is added, and the learning cost is high.
On the other hand, it is not conducive to the use of ordinary users

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, apparatus, electronic device, computer readable storage medium for simplifying programming
  • Method, apparatus, electronic device, computer readable storage medium for simplifying programming
  • Method, apparatus, electronic device, computer readable storage medium for simplifying programming

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0027] Please refer to figure 2 , figure 2 It is a flowchart of a method for simplifying programming provided by the first embodiment of the present invention, and the method is applied to a task scheduling platform. The following takes the task scheduling platform as the Apache Spark task scheduling platform as an example. figure 2 The process shown is described in detail, and the method includes:

[0028] Step S110: Obtain a configuration file constructed for the system to be developed, the configuration file including system operations and interfaces used by tasks corresponding to the system to be developed.

[0029] Different configuration files need to be built for different systems to be developed.

[0030]Wherein, the configuration file may include system operations and interfaces used by the tasks corresponding to the system to be developed, and the definition may configure different Hooks and Operators to schedule different tasks required by the system to be dev...

no. 2 example

[0046] Please refer to image 3 , image 3 It is a structural block diagram of an apparatus 400 for simplifying programming provided by the second embodiment of the present invention, which is applied to a task scheduling platform. The following will be image 3 The structure block diagram shown is described, and the shown device includes:

[0047] An acquisition module 410, configured to construct a configuration file for the system to be developed, the configuration file including system operations and interfaces used by tasks corresponding to the system to be developed;

[0048] A checking module 420, configured to check whether the configuration file is legal;

[0049] An execution module 430, configured to generate a DAG dependency based on the configuration file when the verification module judges yes;

[0050] The calling module 440 is configured to call different Operators and Hooks based on the DAG dependency to process the task, wherein the system operation and i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a simplified programming method, a device, an electronic device and a computer-readable storage medium. The method comprises the following steps: a task scheduling platform obtains a configuration file constructed for a system to be developed; the configuration file comprises a system operation and an interface used for tasks corresponding to the system to be developed; thetask scheduling platform obtains a configuration file constructed for the system to be developed. Verifying whether the configuration file has legitimacy; when yes, generating DAG dependencies based on the configuration file; different Operators and Hooks are called based on the DAG dependency relationship to process the tasks, wherein a plurality of Hooks and a plurality of Operators are configured by the system operation and interface. On the basis of editing the configuration file, the fast processing of the big data related tasks can be realized, the coding is reduced, the development efficiency is improved, and a large amount of human cost is saved. At the same time, the process of continuous development, debugging and correction in the development process is reduced, and the programming task is simplified.

Description

technical field [0001] The present invention relates to the field of data processing, in particular to a method, device, electronic equipment, and computer-readable storage medium for simplifying programming. Background technique [0002] With the development of cloud computing and big data technology, the use of big data technology to process massive data and obtain useful potential information from the data has attracted widespread attention. Spark), to realize offline and real-time processing of massive data. At present, the Apache Spark-based distributed processing platform mainly used provides a series of programming models and corresponding interfaces, and supports mainstream languages ​​such as JAVA / Scala / Python. This distributed cluster has high processing efficiency, strong fault tolerance, high flexibility, and adaptability. It is wide enough to handle various relational and non-relational data processing. It provides technical feasibility for us to process data ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F8/41
CPCG06F8/443
Inventor 王彬
Owner 深圳市木瓜移动科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap