Version publishing method and device based on multi-environment offline task

A multi-environment and task technology, applied in the Internet field, can solve the problems of high development cost of offline tasks, inability to solve business needs in multiple environments, low efficiency, etc., and achieve the effect of saving machine resource consumption.

Active Publication Date: 2020-03-31
深圳市云网万店科技有限公司
View PDF10 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In the era of DT (data processing technology), big data processing technology needs to solve the scalability and complexity of the system. The big data platform carries thousands of ETL task scheduling every day. These tasks have various forms and are single. The tasks in the environment can no longer meet the business needs of multiple environments, which will cause users to repeatedly perform the development, testing, release, and operation and maintenance of offline tasks in different environments, which will easily cause waste of machine resources and human resources. Ultimately, the development cost of offline tasks is too high and the efficiency is low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Version publishing method and device based on multi-environment offline task
  • Version publishing method and device based on multi-environment offline task
  • Version publishing method and device based on multi-environment offline task

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0052] see Figure 1 to Figure 3 , this embodiment provides a method for releasing versions based on multi-environment offline tasks, including:

[0053] Create a new task flow in the development environment and create multiple task sub-nodes, and at the same time obtain the dependencies between the task sub-nodes. The task sub-nodes include task metadata information and task node file resources; upload the task metadata information to the development environment. Save in the database, and upload the task node file resources to the distributed file system in the development environment for storage; obtain the task flow reference version from the target environment, when the task sub-nodes and / or dependencies in the newly created task flow are compared with When the task flow reference version changes, update the version number of the newly created task flow and submit it; obtain the task metadata information corresponding to the task flow with the new version number from the d...

Embodiment 2

[0090] This embodiment provides a version release device based on multi-environment offline tasks, including:

[0091] A development unit, configured to create a plurality of task sub-nodes based on the newly created task flow, and at the same time obtain dependencies among the task sub-nodes, the task sub-nodes include metadata information and node file resources;

[0092] An upload unit, configured to upload the metadata information to a database for storage, and upload the node file resources to a distributed file system for storage;

[0093] A comparing unit, configured to obtain a task flow reference version from the target environment, and update the newly created task flow when the task sub-nodes and / or dependencies in the newly created task flow are changed compared with the task flow reference version version number of

[0094] The publishing unit is used to obtain metadata information corresponding to the task flow with a new version number from the database and pub...

Embodiment 3

[0099] This embodiment provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of the above-mentioned version release method based on multi-environment offline tasks are executed.

[0100] Compared with the prior art, the beneficial effect of the computer-readable storage medium provided by this embodiment is the same as that of the multi-environment offline task-based version release method provided by the above technical solution, and details are not repeated here.

[0101] Those of ordinary skill in the art can understand that all or part of the steps in the above-mentioned inventive method can be completed by instructing related hardware through a program. The above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it includes: For each step of the method in the foregoing embodiments, the foregoing storage m...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a version publishing method and a device based on a multi-environment offline task. The cross-environment publishing efficiency of a task flow can be improved while the cross-environment publishing cost of the task flow is reduced. The method comprises the following steps: creating a new task flow in a development environment, creating a plurality of task sub-nodes, and meanwhile, obtaining a dependency relationship among the task sub-nodes; obtaining a task flow reference version from the target environment, and when the task child nodes and / or dependency relationshipsin the newly-built task flow are / is changed relative to the task flow reference version, updating the version number of the newly-built task flow and then submitting the newly-built task flow; obtaining task metadata information corresponding to the new version number task flow, meanwhile, publishing the task metadata information to a target environment; and obtaining a task node file resource corresponding to the new version number task flow, and triggering a file asynchronous transfer service to publish the task node file resource to the target environment when the task node file resource is changed, thereby realizing publishing of the new version number task flow in the target environment.

Description

technical field [0001] The present invention relates to the technical field of the Internet, in particular to a method and device for releasing versions based on multi-environment offline tasks. Background technique [0002] Offline calculation refers to a calculation mode in which all the data to be calculated is obtained before the calculation task starts, and the amount of data does not change, and the calculation result can be obtained immediately after solving a problem. Offline calculation is usually It is used to perform complex batch calculations on a large amount of data, and has the advantages of large amount of calculation, fast calculation speed, and quick query of batch calculation results. With the rapid development and application of information technology, the scale of industrial application systems is rapidly expanding, and the data generated by it is increasing rapidly. The processing capacity of traditional computing technology and information systems can ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F8/658G06F8/71
CPCG06F8/658G06F8/71
Inventor 曹海洋徐伟王振振李成
Owner 深圳市云网万店科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap