Method and system for information collection and arrangement based on Internet big data

Abstract

The invention relates to a method and system for collecting and organizing information based on Internet big data, wherein, before the system works, the system needs to be built. Set up at least two or more clients, embed a source-end data acquisition terminal in each client, embed a data preprocessing module in the source-end data acquisition end; set up a data server, and configure the data server to have a data server The cache end and the shared cache end, and then establish a communication connection between the client and the data server. The communication connection between the client and the data server adopts a communication wake-up and use method; that is, the data preprocessing module wakes up the connection between the communication channel and the data server through the synchronous task processing thread. The above-mentioned target data can be traced to the source, and all the target data are cached and stored in a shared manner, with strong storage capacity, support for modification at any time, and more timely response to data changes and global flexibility.

Description

technical field [0001] The invention relates to the technical field of Internet big data, in particular to a method and system for collecting and organizing information based on Internet big data. Background technique [0002] Traditional informatization construction is basically business-oriented, and the original manual process is automated and efficient with computer technology. These information systems are often composed of application programs and their corresponding database systems. They are not connected to each other, and gradually form islands of data within the enterprise. [0003] With the increasing update of the Internet of Things and the continuous mining of big data, in some fields, data exchange and integration between data islands have begun to be realized, so as to provide more data-based opinions that can be referred to by different enterprises , when performing data mining, it is often necessary to establish a tracking system between the collection end...

AI Technical Summary

Problems solved by technology

Moreover, common tools are based on the mode of batch and regular data sampling, which is difficult to implement some business scenarios that require high real-time data, and there is no reusability. Every time there is a new demand or the demand changes, it will cost a lot of money Manpower to re-synchronize or modify the synchronization program to achieve

Images