Unlock instant, AI-driven research and patent intelligence for your innovation.

Implementation method for effectively collecting multiple system big data

An implementation method and multi-system technology, applied in the field of efficient collection of multi-system big data, can solve the problems of difficulty in further development of big data, poor data availability, and inflexible configuration methods.

Inactive Publication Date: 2017-03-08
JINAN INSPUR HIGH TECH TECH DEV CO LTD
View PDF1 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Before analyzing and using big data, we must collect big data. However, in the existing data extraction methods, the shortcomings of low data extraction efficiency, poor availability of extracted data, and inflexible supported configuration methods need to be optimized urgently. The shortcomings in data collection also make it difficult for us to further develop in the application of big data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Implementation method for effectively collecting multiple system big data
  • Implementation method for effectively collecting multiple system big data
  • Implementation method for effectively collecting multiple system big data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0016] An implementation method for efficiently collecting multi-system big data includes the following contents and steps:

[0017] A. The overall workflow of multi-system big data collection, as attached figure 1 shown.

[0018] B. Rapid establishment of multi-system database links.

[0019] 1. Design a database link information setting table to record all database link configurations.

[0020]

[0021] C. Realization of accurate data positioning

[0022] The design table data extraction table setting table is used for recording, the structure of the table is as follows:

[0023]

[0024] D. Data extraction optimization processing

[0025] Speed ​​up the comprehensive extraction of all data in the fields in the table by deleting the table index. Rebuild the index after the data extraction is complete.

[0026] Part of the algorithm is as follows:

[0027]

[0028]

[0029] E. Realization of data classification, time-sharing, and batch processing

[0030] ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention provides an implementation method for effectively collecting multiple system big data, and belongs to the field of databases. The method comprises: supporting multiple database link modes of multiple system databases, and establishing a database link information setting table during data extraction to record detailed link configuration information of the multiple system databases; during the data extraction of the multiple system databases, establishing a data extraction table setting table to record detailed information of data extraction; deleting a table index that affects an extraction speed before extracting data; and by classifying data update speeds and in combination with a filtration condition field in the data extraction table setting table, extracting different data at different time, in different batches and under different filtration conditions, so as to realize effective extraction of a large volume of data. Compared with the prior art, the method provided by the present invention has the characteristics of a large data collection quantity, a fast collection speed, and high availability of collected data and the like.

Description

technical field [0001] The invention relates to the field of databases, in particular to an implementation method for efficiently collecting multi-system big data. Background technique [0002] Big data refers to a collection of data that cannot be captured, managed and processed by conventional software tools within a certain period of time. It requires a new processing model to have stronger decision-making power, insights and process optimization capabilities to adapt to Massive, high growth rate and diverse information assets. Before analyzing and using big data, we must collect big data. However, in the existing data extraction methods, the shortcomings of low data extraction efficiency, poor availability of extracted data, and inflexible supported configuration methods need to be optimized urgently. The shortcomings in data collection also make it difficult for us to further develop in the application of big data. Contents of the invention [0003] In order to solv...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/2433G06F16/2282
Inventor 王涛威马娟于超李冉冉
Owner JINAN INSPUR HIGH TECH TECH DEV CO LTD