Data processing method, device and system and storage medium

A technology of data processing and system time, applied in the field of data processing

Active Publication Date: 2020-05-19
CHINA MOBILE SUZHOU SOFTWARE TECH CO LTD +1
View PDF2 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The embodiment of the present invention provides a data processing method, device, system and storage medium, which solves the problem of non-stop service upgrade of the ETL system, reduces data processing delay, and improves the reliability of the ETL system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method, device and system and storage medium
  • Data processing method, device and system and storage medium
  • Data processing method, device and system and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0089] In order to solve the problem of non-stop upgrade of the ETL system, reduce data processing delay, and improve the reliability of the ETL system, the embodiments of the present invention provide a data processing method, device, system and storage medium.

[0090] It should be noted that the non-stop upgrade in the embodiment of the present invention does not mean that the operation of the ETL system is not stopped during the upgrade operation. Rather, it refers to periodically scheduled executions that do not affect already released processes during an upgrade.

[0091] The preferred embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention, and in the absence of conflict, the present invention The embodiments and the features in the embo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method, device and system and a storage medium, solves the problem that an ETL system is upgraded without stopping service, reduces data processing delay andimproves the reliability of the ETL system. The data processing method comprises the following steps: judging whether a missed scheduling period exists or not according to the current system time andthe next scheduling time recorded in the task scheduling table after the service is started; if it is judged that there is a missed scheduling period, adding a scheduling period to the next schedulingtime in the task scheduling table; adding 1 to the maintained missing period counting parameter; and returning to execute the step of judging whether a missed scheduling period exists or not according to the current system time and the next scheduling time recorded in the task scheduling table until the obtained next scheduling time is greater than the current system time, and submitting an execution task to the execution engine according to the missed period count.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a data processing method, device, system and storage medium. Background technique [0002] ETL (Extract-Transform-Load, data extraction, transformation and loading) is an important part of building a data warehouse. Users extract the required data from the data source, after data cleaning, and finally according to the pre-defined data warehouse model, the The data is loaded into the data warehouse. The current ETL tools basically have built-in integration or use third-party scheduling tools to realize their scheduling functions. Under normal circumstances, they can meet the needs of basic process job cycle automatic scheduling. However, when the ETL system fails or the ETL system is upgraded, it is inevitable that there will be an operation to stop the ETL system service. At this time, the service cycle scheduling will be interrupted, which will have a great imp...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/25
Inventor 涂敬伟
Owner CHINA MOBILE SUZHOU SOFTWARE TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap