Netty-based mass data access method and system in Internet of Things application

Abstract

The invention belongs to the technical field of Internet of Things data access, and particularly relates to a Netty-based mass data access method and system in Internet of Things application, and the method comprises the steps: after a Netty server of an access layer node receives a terminal message, identifying the type of the message, and storing the message in a local blocking queue through thread lock-free scheduling; the exclusive consumption thread of the blocking queue pulls the message data from the queue, and if the message is core data, the thread calls a load balancing module to select the message data for forwarding; otherwise, the thread sends the message data to the kafka forwarding exclusive Topic partition through the local Kafka client; the Netty server side of the data analysis layer node receives or pulls the message data from the Kafka, converts the message data into structured data, and sends the structured data to the Kafka business exclusive Topic partition. According to the invention, the advantage that Netty can undertake hundreds of thousands of terminal communication is fully exerted, and service decoupling is realized.

Description

technical field [0001] The invention belongs to the technical field of Internet of Things data access, in particular to a Netty-based massive data access method and system in Internet of Things applications. Background technique [0002] As an important part of the wave of the information industry, the Internet of Things continues to advance its technology and continues to promote the soaring of the Internet of Things industry. The application of the Internet of Things is gradually spreading in various fields of daily life, such as logistics and transportation, health care, smart home, smart industry, smart agriculture etc. Therefore, the number of IoT terminals is growing explosively. How to deal with the massive data access and processing brought by terminals is an urgent problem to be solved in the IoT application industry. [0003] In response to the above problems, existing research usually uses a network framework based on NIO multiplexing technology to solve the prob...

AI Technical Summary

Problems solved by technology

[0004] A major feature of the Netty framework is that a single machine can undertake hundreds of thousands of terminal connections, but in the current research, the service node where Netty is located usually handles terminal message uploads and complex analysis and other business processing, encroaching on the servers available to Netty resources, unable to give full play to the advantages of Netty

At the same time, only relying on Netty cannot completely deal with the problem of massive data access, and data access with burst traffic will overwhelm downstream business processing nodes

Images