Data development method based on data middle station

A data and middle-end technology, applied in the computer field, can solve the problems of unmatched development speed, slow changes in core data models, and inability to keep up with responsiveness.

Pending Publication Date: 2021-04-27
科技谷(厦门)信息技术有限公司
View PDF2 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] As the Internet enters the era of big data, the types of data sources begin to diversify. In data development, the change of core data models is relatively slow, and the workload of data maintenance is also very large. However, the speed of business innovation and the impact on Changes in data requirements are very fast, and this mismatch in development speed has led to a problem that the responsiveness between data development and application development cannot keep up.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data development method based on data middle station

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0023] A data development method based on a data center, comprising the following steps:

[0024] S1. The data source database connects to the target database through the intermediate transmission carrier FlinkX;

[0025] S2. FlinkX periodically synchronizes the data in the data source database to the target database, and performs periodic cleaning;

[0026] The FlinkX uses a star data link when synchronizing data; here FlinkX turns the original complex mesh synchronization link into a star data link, and FlinkX is used as an intermediate transmission carrier to connect various data sources. When accessing a new data source, you only need to connect this data source to FlinkX to achieve seamless data synchronization with the existing data source.

[0027] The periodicity is to set the synchronization time and cleaning time of each day. When the synchronization time starts, the data of the previous day is synchronized, and when the cleaning time starts, the data is cleaned; da...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data development method based on a data center station. The data development method comprises the following steps that S1, a data source library is in butt joint with a target database through an intermediate transmission carrier FlinkX; s2, the FlinkX periodically synchronizes the data in the data source library to a target database, and periodically cleans the data; s3, the data is stored, calculated and converted into a data table, a Spark calculation engine is adopted for calculation of the offline data, and an Flink calculation engine is adopted for calculation of the real-time data; s4, the calculated data table is converted into a data API; and S5, the data middle station issues a data API to an API market through the API gateway, or a user directly calls the data API through the API gateway. Based on the data center station, the data of multiple data sources is synchronized according to the offline data and the real-time data and then converted into the data API, so that the data development time of developers is shortened, the data development threshold is reduced, and the data development efficiency is improved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a data development method based on a data center. Background technique [0002] As the Internet enters the era of big data, the types of data sources begin to diversify. In data development, the change of core data models is relatively slow, and the workload of data maintenance is also very large. However, the speed of business innovation and the impact on Changes in data requirements are very rapid, and this mismatch in development speed has led to a problem that the responsiveness between data development and application development cannot keep up. Contents of the invention [0003] In order to solve the above problems, the present invention provides a data development method based on a data center. [0004] The present invention adopts following technical scheme: [0005] A data development method based on a data center, comprising the following steps: [0006] S1. The ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/27G06F16/215G06F16/25
CPCG06F16/27G06F16/215G06F16/25
Inventor 廖雅哲
Owner 科技谷(厦门)信息技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap