Unlock instant, AI-driven research and patent intelligence for your innovation.

Method, device, electronic device, computer-readable storage medium for simplifying programming

A syntax and legal technology, applied in the field of data processing, can solve the problems of increasing learning cost, high learning cost, and unfavorable use by ordinary users, and achieve the effect of improving development efficiency, reducing coding, and simplifying programming tasks

Active Publication Date: 2021-09-24
深圳市木瓜移动科技有限公司
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The big data (Apache Spark) platform has the characteristics of high flexibility and strong programmability, but it also increases the learning cost for our ordinary programming and operation and maintenance personnel while being flexible.
Mainly in daily business, if you need to add new statistical tasks, you need to rewrite the interface to complete code editing, compiling and debugging. On the one hand, relatively speaking, a lot of repetitive and unnecessary work is added, and the learning cost is high.
On the other hand, it is not conducive to the use of ordinary users

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, device, electronic device, computer-readable storage medium for simplifying programming
  • Method, device, electronic device, computer-readable storage medium for simplifying programming
  • Method, device, electronic device, computer-readable storage medium for simplifying programming

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0027] Please refer to figure 2 , figure 2 It is a flow chart of a simplified programming method according to a first embodiment of the present invention, the method being applied to a task scheduling platform. The following is an example of the Apache Spark task scheduling platform with the task scheduling platform. figure 2 The flow shown in detail, the method includes:

[0028] Step S110: Gets the configuration file built for the system to be developed, the configuration file including the system operation and interface used with the task corresponding to the system to be developed.

[0029] The configuration files need to be constructed are different for different main development systems.

[0030]Wherein, the configuration file can include a system operation and an interface used with the task corresponding to the to be developed, which can configure different hook and operator to schedule different task job processing required to be developed.

[0031] Of course, optional, t...

no. 2 example

[0046] Please refer to image 3 , image 3 It is a structural block diagram of a simplified programmed device 400 according to a second embodiment of the present invention, applied to the task scheduling platform. Below image 3 The structural block diagram shown is illustrated, and the apparatus shown:

[0047] The module 410 is used to configure a configuration file to be built to be developed, the configuration file including system operation and interface used by the task corresponding to the system to be developed;

[0048] The calibration module 420 is used to verify that the configuration file has legitimacy;

[0049] The execution module 430 is used to determine whether the DAG dependence is generated based on the configuration file at the verification module.

[0050] Calling module 440 for calling different Operator and Hook based on the DAG dependence, wherein the system operation and interface configuring a plurality of hooks and multiple Operator.

[0051] In this embodi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention provides a method, device, electronic device, and computer-readable storage medium for simplifying programming. The method includes: the task scheduling platform acquires a configuration file constructed for the system to be developed, and the configuration file includes the The system operation and interface used by the corresponding task of the system; verify whether the configuration file is legal; if yes, generate DAG dependencies based on the configuration file; call different Operators and Hooks based on the DAG dependencies , to process the task, wherein the system operation and interface are configured with multiple Hooks and multiple Operators. On the basis of editing configuration files, the rapid processing of big data-related tasks can be realized, which reduces coding, improves development efficiency, saves a lot of labor costs, and reduces the process of continuous development, debugging and correction during the development process. Programming tasks are simplified.

Description

Technical field [0001] The present invention relates to the field of data processing, and in particular, a method, apparatus, electronic device, computer readable storage medium, a simplified programming. Background technique [0002] With the development of cloud computing, large data technology, using large data technology to deal with massive data and obtain useful potential information from data, it is widely concerned. The current general program is based on open source distributed big data processing framework (Apache) Spark), realizing offline and real-time processing of massive data. At present, the Apachespark distributed processing platform is currently provided with a series of programming models and corresponding interfaces, supporting mainstream languages ​​such as Java / Scala / Python. This distributed cluster has high efficiency, strong fault tolerance, high flexibility, adaptation surface Wide, meet the processing of various relationships and non-relational data ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F8/41
CPCG06F8/443
Inventor 王彬
Owner 深圳市木瓜移动科技有限公司