Configuration-driven lightweight batch data processing method and device
A batch data and processing method technology, applied in the direction of electrical digital data processing, program control device, program loading/starting, etc., can solve the problems of incomplete modification and importing
Pending Publication Date: 2021-06-22
CCB FINTECH CO LTD
View PDF6 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] The embodiment of the present invention provides a configuration-driven lightweight batch data processing method, which is used to solve the problem that when there is a batch modification content, it is necessary to modify the scattered source code, which is prone to incomplete modification and more problems. Methods include:
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0031] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments and descriptions of the present invention are used to explain the present invention, but not to limit the present invention.
[0032] Explanation of terms:
[0033] Configuration file: the file containing the input configuration parameters during system operation, usually in XML, JSON and excel file formats that are easy to edit;
[0034] Task: In order to solve a certain problem, it is a collection of multiple sequential steps. Generally, a task will be represented by a task configuration file;
[0035] Atomic operations: logically indivisible operations from program processing;
[0036] Step: An atomic operation in a task. The steps to deal with a class of problems are stored in the same...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a configuration-driven lightweight batch data processing method and device, and relates to the field of automatic programming. The method comprises the steps: initializing an environment variable; based on the name of the job configuration file, collecting the configured job configuration file, and processing step by step; step information which is required for completing a task and has a sequence is configured in the job configuration file; a corresponding defined processor and a configured step configuration file are searched based on the information of each step, the defined processor executes the corresponding step, and operation parameters of each step are configured in the step configuration file; based on the configured step configuration file, operating parameters of the corresponding steps are injected into a defined processor; collecting corresponding service data based on the service date; and running the defined processor based on the service data until all the steps are processed. By configuring the driven lightweight batch processing framework, batch data processing work can be completed in a short time.
Description
technical field [0001] The invention relates to the technical field of batch data processing in the field of automatic program design, in particular to a configuration-driven lightweight batch data processing method and device. Background technique [0002] This section is intended to provide a background or context to embodiments of the invention that are recited in the claims. The descriptions herein are not admitted to be prior art by inclusion in this section. [0003] Most of the existing batch processing scenarios are implemented by writing code, which has high flexibility, but the ability of coders is uneven, which is easy to introduce new problems and lead to code redundancy. When there is a batch modification of the content, it is necessary to modify the scattered source code, which is prone to incomplete modification and more problems. Contents of the invention [0004] The embodiment of the present invention provides a configuration-driven lightweight batch da...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G06F9/445G06F8/20G06F8/41
CPCG06F9/4451G06F8/24G06F8/44
Inventor 杨有为刘庆文
Owner CCB FINTECH CO LTD