Distributed high-concurrency data collection and processing method and system in Internet of Things environment

Abstract

The invention discloses a distributed high-concurrency data collection and processing method and system in an Internet of Things environment. The method comprises the following steps of: ; the methodcomprises the following steps of: creating a process guide Server Bootstrap; the method comprises the following steps: initializing a thread group bossGroup and a thread group workerGroup of which thetypes are NioEventLoopGroup, and initializing a thread group BossGroup and a thread group workerGroup of which the types are NioEventLoopGroup; the method comprises the following steps of: starting adata transponder process guider of which the type is Bootstrap; the method comprises the following steps of: constructing a processing class which is inherited from a ChannelInbound Handler Adapter;and adding processors Handler for different protocols, constructing a main interceptor, and binding a process port by utilizing the data receiving guider constructed in the above steps to monitor datasent by hardware equipment and analyze and transmit the data to a message queue unit. According to the distributed high-concurrency data collection and processing method and system in the Internet ofThings environment, a Netty framework is adopted, and large-batch data collection is supported.

Description

technical field [0001] The present invention relates to the technical field of industrial Internet of Things data collection under big data, in particular to a distributed high-concurrency data collection and processing method and system under the Internet of Things environment. Background technique [0002] For most manufacturing enterprises, the automatic data collection of measuring instruments and operating equipment has always been an annoying thing. Not only is the work heavy, but the accuracy of the data cannot be guaranteed, and the data often lags behind; How to achieve high-efficiency, concise and real-time data collection is a big problem. [0003] The data collection of big data is based on the determination of the user's goals, and aims at the collection of all structured, semi-structured and unstructured data within the scope, and processes these data after collection, and analyzes and excavates valuable information from them. Information. In the process of b...

AI Technical Summary

Problems solved by technology

In the process of big data collection, there are two main challenges. One is the diversity of communication protocols and data protocols of heterogeneous IoT devices. In addition, with the access of massive devices, the high concurrency of data collection will increase Generate performance bottlenecks, leading to problems such as data backlog and connection timeout

Images