Unlock instant, AI-driven research and patent intelligence for your innovation.

Big data extraction method and device, computer equipment and storage medium

A big data and computer program technology, applied in database indexing, digital data protection, structured data retrieval, etc., can solve problems such as reducing data extraction efficiency and achieve the effect of improving efficiency

Pending Publication Date: 2022-04-08
南方电网大数据服务有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, in the traditional method, when the big data platform connects to a target business system of a different database type, it is necessary to write data collection scripts corresponding to each database type, so that the big data platform can extract data from different types of target business system databases. data, thus reducing the efficiency of data extraction to a certain extent

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data extraction method and device, computer equipment and storage medium
  • Big data extraction method and device, computer equipment and storage medium
  • Big data extraction method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0057] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

[0058] The big data extraction method provided by the embodiment of this application can be applied to such as figure 1 shown in the application environment. Among them, it includes a big data platform 10 and several business systems 20. The big data platform 10 is used as an enterprise-level server. The big data platform 10 can be a Hadoop distributed file system. Each business system 20 is a database that stores data in different business scenarios, including specific Data in business scenarios. For example, the sales business system 20 generates a large amount of sale...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a big data extraction method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a system category and table information of a target service system; determining the database type of the target business system according to a first corresponding relation between a preset system category and the database type and the system category of the target business system; determining a target database field type of the target business system according to the database type of the target business system; according to the target database field type of the target business system and the table information of the target business system, performing extraction through a data dictionary extraction script to generate a data dictionary of the target business system; and reading data in the target service system through a preset data acquisition script and the data dictionary of the target service system. By adopting the method, the data extraction efficiency can be improved.

Description

technical field [0001] The present application relates to the technical field of data storage, and in particular to a big data extraction method, device, computer equipment, storage medium and computer program product. Background technique [0002] With the development of data storage technology, in order to realize the transfer and storage of massive data, such as storing the business data of various business systems on the big data platform, big data extraction technology has emerged. [0003] In the traditional technology, when the big data platform accesses the target business system, it usually determines the database type of the target business system from the operation and maintenance manufacturer during the demand research, and writes the corresponding data collection script according to the database type of the target business system. When the data platform executes the corresponding data collection script, it extracts the corresponding data from the database of the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/25G06F16/22G06F21/60G06F21/62
Inventor 赵永国杨荣霞曹熙曾祥清黎名航
Owner 南方电网大数据服务有限公司