Internet of Things platform based on side cloud collaboration

Abstract

The invention discloses an Internet of Things platform based on side cloud collaboration, and belongs to the technical field of Internet of Things platforms. Comprising a plurality of data docking micro-service modules, a message queue module, a data processing micro-service module, a data application micro-service module and a data interaction micro-service module. Each data docking micro-service module is independently docked with an AIOT equipment protocol; the data processing micro-service module processes the received data to obtain final data; and finally, distributing the final data to the data application micro-service module and / or the data interaction micro-service module according to the data purpose. And a micro-service architecture is adopted, so that the system is modularized, and independent development, testing, deployment and expansion are facilitated. Each protocol supporting heartbeat connection is independent as a micro-service, so that the coupling degree of the system is reduced, and the overall maintainability and reliability are improved. The design enhances the expansibility of the system at the same time, so that newly added functions or upgrading existing functions are more flexible and efficient.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things platforms, and in particular to an Internet of Things platform based on edge-cloud collaboration. Background Art

[0002] With the rapid development of IoT technology, traditional IoT platforms mainly focus on supporting various types of sensor devices (such as human detection, gas detection, and smoke detection), and usually use MQTT or HTTP protocols for data transmission. However, these traditional platforms have obvious limitations in the scope of device support and business functions. They usually cannot meet the access requirements of multiple devices such as edge AI computing devices and cameras, and are difficult to adapt to the integration of different protocols (such as GB28181, RTSP) and multi-vendor SDKs. In addition, traditional platforms cannot implement specific protocol communications with the intranet, because many protocols require the server to actively initiate requests but the device does not have a public IP, making the management and control of intranet devices difficult.

[0003] In this context, traditional IoT platforms often only focus on data collection, lack the function of in-depth "consumption" of device data, and cannot effectively support complex business scenarios such as elderly care, fire warning, and parking space management. These limitations lead to the following key issues: 1. Inadequate architecture design: Although multi-protocol and multi-SDK support can be achieved, poor architecture design makes it difficult for the system to maintain stability and reliability in high-concurrency scenarios, which may lead to performance bottlenecks.

[0004] 2. Weak business data consumption capabilities: Existing platforms have insufficient "consumption" capabilities for device data and lack the ability to analyze and respond to real-time data, which limits the implementation of complex business functions and their actual application effects.

[0005] 3. High coupling and low cohesion: Many traditional platforms have high coupling in functional implementation, which leads to system complexity and increases the difficulty of subsequent maintenance. This design defect not only affects the flexibility of the system, but also increases the cost of development and project management, and cannot meet the growing application needs.

[0006] 4. Lack of high availability and load balancing: Traditional platforms often fail to effectively achieve high availability, have insufficient response capabilities when facing failures, lack the ability to design for resilience and fault-oriented programming, and thus affect system reliability.

[0007] 5. Insufficient SaaS capabilities: Traditional platforms often lack the ability to provide SaaS services. In addition to the system's carrying capacity, they also lack features such as convenient user access, centralized management, automatic updates and maintenance, and on-demand billing. These deficiencies make it difficult for users to access services in different locations, increase the company's investment in infrastructure and operation and maintenance, and limit the flexibility and scalability of the business. Summary of the invention

[0008] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an Internet of Things platform based on edge-cloud collaboration.

[0009] The object of the present invention is to achieve the following technical solutions: an Internet of Things platform based on edge-cloud collaboration, comprising a plurality of data docking microservice modules, wherein the plurality of data docking microservice modules are all connected to a message queue module; the message queue module is connected to a data processing microservice module; the data processing microservice module is connected to a data application microservice module and a data interaction microservice module; the data application microservice module is connected to a data interaction microservice module; each data docking microservice module independently docks an AIOT device protocol; When receiving external data, first perform structure parsing, convert the original data bytes into an internally recognizable target data format, and filter out unnecessary data segments; then filter the data by the value of a specific field to obtain the first data; then send the first data to the data processing microservice module through the message queue module; After receiving the first data, the data processing microservice module converts the first data from the original protocol-specific format to the platform's general data format, and then performs field value conversion again; then, the alarm data is deduplicated, and based on the predefined configuration rules, it is determined whether it is a duplicate alarm, and the duplicate alarm data is deleted; then, it is detected whether there is data missing. If there is data missing, the mapping information in the database is used to query and complete the missing data fields to obtain the final data; finally, the final data is distributed to the data application microservice module and / or the data interaction microservice module according to the data usage.

[0010] Preferably, each data docking microservice module, data processing microservice module, data application microservice module, and data interaction microservice module are independent of each other, their functions are independently developed, they are all stateless applications, and their data are stored in an external database, and they do not share the external database with each other.

[0011] Preferably, for AIOT device protocols that do not support heartbeat connections, an Agent in the intranet is used to interact with the AIOT device, and then a private protocol is used to enable communication between the Server and the Agent.

[0012] Preferably, a multi-tenant architecture is adopted, and users select functional modules as needed.

[0013] Preferably, communicating via a private protocol comprises the following steps: The intranet agent actively initiates a TCP connection to the public network server; The Agent sends a heartbeat packet. The Server monitors the Agent's connection status based on the frequency of the received heartbeat packets. If no heartbeat is received within the set time, the Server considers the connection to be disconnected and attempts to reconnect. The agent sends a request, and the server processes and responds to the request. All data transmitted through the TCP connection is encrypted using the TLS protocol. During the TCP connection establishment phase, the agent sends authentication information including an authentication token. The Agent and Server monitor the communication status in real time and log any abnormal status. The Agent automatically tries to reconnect when it detects a connection interruption, and the Server actively tries to reestablish the connection after the heartbeat timeout period has expired. The Agent and Server record detailed logs of all requests, responses, and heartbeats and upload them to the monitoring system.

[0014] Preferably, the specific field is an alarm status field.

[0015] Preferably, the predefined configuration rules are time windows and time types.

[0016] The beneficial effects of the present invention are: 1) The microservice architecture makes the system modular and easy to develop, test, deploy and expand independently. Each protocol that supports the heartbeat connection is an independent microservice, which reduces the coupling of the system and improves the overall maintainability and reliability. This design also enhances the scalability of the system, making it more flexible and efficient to add new functions or upgrade existing functions.

[0017] 2) Implement edge-cloud collaborative architecture, deploy Agent in the intranet, interact directly with edge devices, realize preliminary data processing and quickly respond to local events. This mechanism significantly improves the real-time and accuracy of data processing, while reducing the data processing burden on the cloud.

[0018] 3) The platform provides SaaS services, supports multi-tenant architecture and on-demand billing, allowing users to use services without a large initial investment. Automated system updates and maintenance reduce the operating costs of enterprises and improve user experience. Centralized management and convenient user access interface further enhance user convenience.

[0019] 4) Through the specific microservice architecture and edge-cloud collaboration, the high performance and high availability of the system are ensured, while the effective consumption and intelligent processing of business data are realized, providing an efficient, reliable and cost-effective solution that adapts to the comprehensive requirements of modern IoT applications for high efficiency, high reliability and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is the architecture diagram of the Internet of Things platform based on edge-cloud collaboration. DETAILED DESCRIPTION

[0021] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] See also Figure 1 The present invention provides a technical solution: an Internet of Things platform based on edge-cloud collaboration, comprising a plurality of data docking microservice modules, wherein the plurality of data docking microservice modules are all connected to a message queue module; the message queue module is connected to a data processing microservice module; the data processing microservice module is connected to a data application microservice module and a data interaction microservice module; the data application microservice module is connected to a data interaction microservice module; each data docking microservice module independently docks an AIOT device protocol; When receiving external data, first perform structure parsing, convert the original data bytes into an internally recognizable target data format, and filter out unnecessary data segments; then filter the data by the value of a specific field to obtain the first data; then send the first data to the data processing microservice module through the message queue module; After receiving the first data, the data processing microservice module converts the first data from the original protocol-specific format to the platform's general data format, and then performs field value conversion again; then, the alarm data is deduplicated, and based on the predefined configuration rules, it is determined whether it is a duplicate alarm, and the duplicate alarm data is deleted; then, it is detected whether there is data missing. If there is data missing, the mapping information in the database is used to query and complete the missing data fields to obtain the final data; finally, the final data is distributed to the data application microservice module and / or the data interaction microservice module according to the data usage.

[0023] In this embodiment, for protocols that support heartbeat connections (such as GB28181, WebSocket) and protocols that are mainly used to receive data (such as HTTP), a dedicated microservice cluster is set up for data interface. Each protocol is handled by an independent microservice to ensure efficient and stable data reception. The received data is first parsed by the structure, the original data bytes are converted into an internally recognizable data format, and unnecessary data segments are filtered out. Further data filtering is performed by the value of a specific field (such as the alarm status field), for example, only data with an alarm status of activated is retained, thereby reducing the invalid burden of subsequent processing. The data that has been preliminarily processed is sent to the data processing microservice through a message queue for more in-depth unified processing. After receiving the data, the data processing microservice first converts it from the original protocol-specific format to the common data format of the platform. This step covers discarding unnecessary format complexity and standardizing the data to simplify subsequent processing. Field value conversion again: perform necessary conversion processing on specific fields, such as converting the date and time field into a unified millisecond timestamp format to facilitate the processing and comparison of time series. Deduplication: Deduplication is performed on alarm data. Based on predefined configuration rules (such as time windows and event types), it is determined whether it is a duplicate alarm to avoid processing the same event multiple times. Data completion: In the case of incomplete data (such as the presence of device ID but the lack of user ID), the mapping information in the database is used to query and complete the missing data fields to ensure data integrity. Data distribution: The processed and completed data is distributed to different microservices according to its purpose. For example, the data application microservice handles display and alarm logic, and the data delivery microservice is responsible for pushing data to external systems or third-party platforms.

[0024] In some embodiments, each data docking microservice module, data processing microservice module, data application microservice module, and data interaction microservice module are independent of each other, their functions are independently developed, they are all stateless applications, data is stored in an external database, and they do not share the external database with each other.

[0025] In this embodiment, the platform adopts a microservice architecture and develops each functional module independently. When docking the protocol of the AIoT device, each protocol is independently a data docking microservice to ensure the independence and flexibility of the protocol. In addition, the functions of data processing and data interaction are also independent microservices. The direct data flow of microservices is through message queues and RPC. Since each microservice is relatively independent, when a microservice cluster crashes, it will not affect the clusters of other protocols. Each container in the cluster will run on a different physical server as much as possible, while ensuring that the physical server fails to avoid the impact on the business. This design enables the system to have stronger performance, flexibility and maintainability when functions change and user needs. High availability and load balancing: All microservices are stateless applications, data is stored in external databases, and microservices try not to share external databases as much as possible to be as independent as possible. This design and concept not only simplifies service management, but also improves scalability, service security and stability. The platform supports deployment on Kubernetes (K8S) to achieve automatic expansion, load balancing and fault self-recovery capabilities through the functions of the container cluster, ensuring stable operation in high concurrency scenarios.

[0026] In some embodiments, for AIOT device protocols that do not support heartbeat connections, an Agent in the intranet is used to interact with the AIOT device, and then a private protocol is used to enable communication between the Server and the Agent.

[0027] In this embodiment, for protocols that do not support heartbeat connections, an Agent in the intranet is used to interact with the device, and then a private protocol is used to enable communication between the Server and the Agent, so that the protocol that does not support heartbeat connections can enable the Server in the cloud server to send data and signaling to the intranet device. Such a private protocol is also an independent microservice. Edge-cloud collaboration: Efficient collaboration between edge computing and cloud computing is achieved through the Agent-Server architecture. The Agent is deployed in the intranet and is responsible for direct interaction with edge devices, collecting data and performing preliminary processing. The problem that the protocol cannot cross the intranet communication is solved. Important information is then sent to the cloud for further analysis. By analyzing the data of edge devices, the cloud realizes intelligent decision-making, responds to various business needs in a timely manner, and improves the real-time processing capabilities and overall intelligence level of the platform.

[0028] In some embodiments, a multi-tenant architecture is adopted and users select functional modules as needed.

[0029] In this embodiment, it has SaaS service capabilities: the platform has a multi-tenant architecture and supports users to access services on demand. Users can choose functional modules according to actual business needs, charge on demand, and reduce enterprise infrastructure investment. Automated updates and maintenance reduce the user's operation and maintenance burden and improve user experience. The centralized management interface provides a simple operation method to enhance user convenience. Each data docking microservice receives and filters data to ensure that only necessary data is processed. These data are transmitted to a unified data processing microservice through a message queue for format conversion and extraction of necessary data to facilitate subsequent use. After processing, the data can be sent to the data application microservice or data interaction microservice within the platform as needed, and finally pushed to a third-party platform or text message, phone, to achieve flexible data consumption and business expansion.

[0030] In some embodiments, communicating via a proprietary protocol includes the following steps: The intranet agent actively initiates a TCP connection to the public network server; The Agent sends a heartbeat packet. The Server monitors the Agent's connection status based on the frequency of the received heartbeat packets. If no heartbeat is received within the set time, the Server considers the connection to be disconnected and attempts to reconnect. The agent sends a request, and the server processes and responds to the request. All data transmitted through the TCP connection is encrypted using the TLS protocol. During the TCP connection establishment phase, the agent sends authentication information including an authentication token. The Agent and Server monitor the communication status in real time and log any abnormal status. The Agent automatically tries to reconnect when it detects a connection interruption, and the Server actively tries to reestablish the connection after the heartbeat timeout period has expired. The Agent and Server record detailed logs of all requests, responses, and heartbeats and upload them to the monitoring system.

[0031] In this embodiment, all data transmitted through the TCP connection is encrypted using the TLS protocol to ensure the security of the data during transmission. The agent and server should monitor the communication status in real time and log any exceptions (such as connection interruption, timeout, etc.). The agent and server record detailed logs of all requests, responses, and heartbeats, including successful responses and exceptions. The monitoring system is used to track and analyze communication data in real time to facilitate problem tracking and system optimization. Heartbeat packet format: message type (message_type): "heartbeat"; Agent identifier (agent_id); timestamp (timestamp).

[0032] Heartbeat response format: message type (message_type): "ack_heartbeat"; timestamp (timestamp).

[0033] Request format: message type (message_type): "request"; request ID (request_id); operation type (operation_type): such as "http_request" or "sdk_request"; request parameters (parameters): depending on the operation type, it contains the detailed parameters required for the specific operation.

[0034] Request response format: message type (message_type): "response"; request ID (request_id); response status (status): such as "success" or "error"; response content (data): the result data of the operation.

[0035] Authentication format: message type (message_type): "auth"; Agent ID (agent_id); verification token (token).

[0036] In some embodiments, the specific field is an alarm status field.

[0037] In some embodiments, the predefined configuration rules are time windows and time types.

[0038] In the present invention, each protocol that supports heartbeat connection is independently a microservice, and non-supported protocols are independently a microservice through a private protocol between the Agent and the Server to achieve support for heartbeat connection, ensuring two-way communication without a public IP address. The unified data processing microservice processes and forwards data through message queues to ensure the efficiency and stability of data flow. The Agent-Server architecture realizes efficient collaboration between edge computing devices and cloud computing platforms, balancing the real-time and cost of data processing and decision-making. The platform provides SaaS services, supports multi-tenant architecture, automated maintenance and updates, and on-demand billing, enhances user access convenience and reduces operating costs.

[0039] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. An IoT platform based on edge-cloud collaboration, characterized by: It includes multiple data docking microservice modules, and each of the multiple data docking microservice modules is connected to a message queue module; The message queue module is connected to the data processing microservice module; The data processing microservice module connects the data application microservice module and the data interaction microservice module; The data application microservice module is connected to the data interaction microservice module; Each data docking microservice module independently docks an AIOT device protocol; When receiving external data, first perform structure parsing, convert the original data bytes into an internally recognizable target data format, and filter out unnecessary data segments; then filter the data by the value of a specific field to obtain the first data; then send the first data to the data processing microservice module through the message queue module; After receiving the first data, the data processing microservice module converts the first data from the original protocol-specific format to the platform general data format, and then performs field value conversion again; then, the alarm data is deduplicated, and based on the predefined configuration rules, it is determined whether it is a duplicate alarm, and the duplicate alarm data is deleted; Then check whether there is any missing data. If there is any missing data, use the mapping information in the database to query and complete the missing data fields to obtain the final data; finally, distribute the final data to the data application microservice module and / or the data interaction microservice module according to the data usage.

2. The edge-cloud collaboration-based IoT platform according to claim 1 is characterized in that: Each data docking microservice module, data processing microservice module, data application microservice module, and data interaction microservice module are independent of each other, and their functions are independently developed. They are all stateless applications, and the data is stored in an external database, and they do not share the external database with each other.

3. The edge-cloud collaboration-based IoT platform according to claim 1 is characterized in that: For AIOT device protocols that do not support heartbeat connections, the Agent in the intranet interacts with the AIOT device, and then the Server and Agent communicate through a private protocol.

4. The edge-cloud collaboration-based IoT platform according to claim 1 is characterized in that: It adopts a multi-tenant architecture and users can select functional modules as needed.

5. The edge-cloud collaboration-based IoT platform according to claim 3 is characterized in that: Communication via a proprietary protocol includes the following steps: The intranet agent actively initiates a TCP connection to the public network server; The Agent sends a heartbeat packet. The Server monitors the Agent's connection status based on the frequency of the received heartbeat packets. If no heartbeat is received within the set time, the Server considers the connection to be disconnected and attempts to reconnect. The agent sends a request, and the server processes and responds to the request. All data transmitted through the TCP connection is encrypted using the TLS protocol. During the TCP connection establishment phase, the agent sends authentication information including an authentication token. The Agent and Server monitor the communication status in real time and log any abnormal status. The Agent automatically tries to reconnect when it detects a connection interruption, and the Server actively tries to reestablish the connection after the heartbeat timeout period has expired. The Agent and Server record detailed logs of all requests, responses, and heartbeats and upload them to the monitoring system.

6. The edge-cloud collaboration-based IoT platform according to claim 1, characterized in that: The specific field is the alarm status field.

7. The IoT platform based on edge-cloud collaboration according to claim 1 is characterized in that: The predefined configuration rules are time window and time type.

Images