Industrial data collection platform and method

Abstract

The present invention provides an industrial data acquisition platform and method, which includes: Ansible control node, which is used to publish data processing tasks, manage all nodes of K3s cluster based on SSH protocol, and realize batch system configuration, program deployment and command operation; K3s main service node, which is used to manage, monitor, maintain and schedule K3s working nodes; K3s working nodes, which are used to perform specific data acquisition and data processing tasks; the K3s working nodes include data acquisition agent nodes and adapter nodes, the agent nodes are deployed on edge networks, and the adapter nodes are deployed on data acquisition devices. Through this solution, it is easy to access, deploy and manage hardware resources, improve horizontal expansion capabilities, and at the same time, dynamically adjust and allocate computing resources to meet industrial data acquisition services.

Description

Technical Field

[0001] The present invention belongs to the field of industrial Internet, and in particular relates to an industrial data collection platform and method. Background Art

[0002] In industrial intelligent manufacturing, there will be a variety of data collection source devices with different functional attributes and hardware components that need to be connected to the network, such as CNC machine tools, industrial robots, automatic transport vehicles, cleaning equipment, testing equipment, automated production lines and automatic material warehouses, as well as industrial ancillary equipment such as controllers for multiple sensors / actuators. For the diverse industrial production data, it is necessary to connect to the Internet for unified storage and analysis in order to monitor, control and adjust the production process.

[0003] However, currently, for multi-source heterogeneous device data, the collected data will be directly sent to the node server for storage and analysis, and coordinated and managed through the main service, combined with data caching, message queues and other technologies to achieve load balancing. This method can better cope with massive data collection tasks, but for the expansion of access to multiple heterogeneous collection devices and collection servers, the server and collection device configuration and deployment processes are very cumbersome. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides an industrial data acquisition platform and method, which are used to solve the problem that the access configuration process of existing extension servers or acquisition devices is relatively complicated.

[0005] In a first aspect of an embodiment of the present invention, an industrial data acquisition platform is provided, including:

[0006] Ansible control node, used to publish data processing tasks, manage all nodes in the K3s cluster based on the SSH protocol, and implement batch system configuration, program deployment, and command execution;

[0007] K3s master service node, used to manage, monitor, maintain and schedule K3s working nodes;

[0008] K3s working node, used to perform specific data collection and data processing tasks;

[0009] The K3s working node includes a data collection agent node and an adapter node. The agent node is deployed on an edge network, and the adapter node is deployed on a data collection device.

[0010] In a second aspect of an embodiment of the present invention, an industrial data collection method is provided, including:

[0011] Release data collection tasks based on Ansible control nodes and manage K3s clusters;

[0012] Manage, monitor, maintain and schedule K3s working nodes through K3s master service nodes, and perform specific data collection and processing tasks through K3s working nodes;

[0013] Industrial equipment data is acquired based on the adapter node and processed through the agent node.

[0014] In the embodiment of the present invention, the containerized K3s platform, as well as the agents and adapters of the working nodes, not only facilitate the management, deployment and configuration of various acquisition devices and acquisition servers, improve the horizontal expansion capability of the acquisition platform, and realize "one-time development, multiple operations", but also can dynamically adjust computing resources and improve the data processing capability of the acquisition platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A schematic diagram of the structure of an industrial data acquisition platform provided by one embodiment of the present invention;

[0017] Figure 2 A schematic diagram of a K3s cluster high-availability service architecture provided for an embodiment of the present invention;

[0018] Figure 3 A schematic diagram of an industrial data collection architecture provided by an embodiment of the present invention;

[0019] Figure 4 A schematic diagram of a communication architecture for a data acquisition model file configuration distribution protocol provided by an embodiment of the present invention;

[0020] Figure 5 A schematic diagram of an industrial data collection microservice architecture provided by an embodiment of the present invention;

[0021] Figure 6 A schematic diagram of the K3s containerized platform architecture provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be understood that the term "including" and other similar expressions in the specification or claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions, such as a process, method, system, or device including a series of steps or units is not limited to the listed steps or units. In addition, "first" and "second" are used to distinguish different objects, not to describe a specific order.

[0024] See also Figure 1 , a schematic diagram of the structure of an industrial data acquisition platform provided by an embodiment of the present invention, including:

[0025] Ansible control node 110 is used to publish data processing tasks, manage all nodes of the K3s cluster based on the SSH protocol, and implement batch system configuration, program deployment and command execution;

[0026] K3s master service node 120, used to manage, monitor, maintain and schedule K3s working nodes;

[0027] K3s working node 130, used to perform corresponding specific data collection and data processing tasks;

[0028] K3s is a lightweight Kubernetes that provides a "one master and multiple slaves" architecture model. The master node (Server) is only the control center of the entire cluster, which is called the Master node in the Kubernetes protocol, that is, the "control plane", and its slave nodes are the real workload nodes. Actual business applications must all be deployed on the slave nodes, so that the characteristics of the cluster infrastructure can be used to exclude the control plane from the business load, making its work scope more specific, so that the entire cluster can have higher performance, availability and stability.

[0029] K3s slave nodes (Agents) are the general name and abstraction of all working nodes. In Kubernetes, they are called Worker nodes, which are the load nodes of business applications. Since the scope of application, performance requirements, software and hardware indicators of each business are different, the slave nodes should be subdivided so that various applications can be deployed to a certain type of slave node in a targeted manner, achieving multi-granular control of cluster node resources.

[0030] like Figure 2 As shown in the figure, in Kubermetes, controllers and service nodes between different virtual machines perform corresponding maintenance and forwarding through Pods.

[0031] The K3s master node is the infrastructure framework of the K3s cluster, which is mainly responsible for the management, monitoring, maintenance, scheduling and other basic services of the entire K3s cluster. In each K3s working node, there is a k3s-agent cluster framework process, which communicates with the K3s master node and acts as an agent for all infrastructure affairs related to the node cluster.

[0032] The K3s master service node contains the configuration information of the K3s cluster, and the K3s master service node sets corresponding Kubernetes labels and stains according to different processing tasks of the working nodes.

[0033] In one embodiment, the Rancher cluster runs on the K3s master service node and is a web application for managing the entire K3s cluster; the Rancher cluster can be deployed in the K3s cluster through Helm for K3s cluster management, application deployment, cluster operation and maintenance, and cluster monitoring. Based on this web application, you can easily handle all transactions of the K3s cluster in the browser, monitor its operating status, perform some temporary tests, adjust parameters, etc., instead of just executing related commands through the console kubectl tool, which is more intuitive and efficient.

[0034] The K3s working node 130 includes a data collection agent node and an adapter node. The agent node is deployed on an edge network, and the adapter node is deployed on a data collection device.

[0035] The adapter-agent model in the software architecture is used to construct the industrial Internet data collection application system. The adapter that directly interacts with the underlying industrial equipment is connected to its data collection interface. As the lowest-level data collection application, the adapter can connect and communicate with a variety of heterogeneous devices, such as Figure 3 shown.

[0036] Optionally, the proxy node includes a file management module and a file distribution module. The file management module implements data transmission between the front-end and back-end of the website and the database through a REST interface, and the file distribution module performs database reading and model file transmission through a gRPC communication protocol. Figure 4 shown.

[0037] The agent node, i.e., the data acquisition server, is a gateway application for the adapter data traffic in the network area that the agent is responsible for. Due to the complexity of the actual application environment of industrial data acquisition and the variety of network architectures, the subnet that an agent is responsible for may include hundreds or thousands of adapters. In order to cope with high concurrent access requirements, message queue middleware such as MQTT can be used to decouple the adapter and the agent, so as to facilitate the deployment of multiple agents and load balancing among multiple agent nodes. Alternatively, the single-point processing capability can be improved by enhancing the hardware performance of the acquisition server.

[0038] Among them, the MQTT subscription-publishing message queue is used as a data transmission middleware service, which can realize the forwarding of edge data collection of the industrial Internet and cope with complex and changeable actual needs. In order to facilitate deployment in the K3s cluster, the implementation form of the open source Eclipse Mosquitto container image is used as the broker proxy server of MQTT. In addition, the architecture design is simplified to a certain extent, and the MQTT broker service and model configuration related services are integrated into one node. In actual applications, if the MQTT traffic is large, a separate node can be set up and deployed in a dedicated manner to avoid competing for resources with other applications and forming a bottleneck for data transmission. Furthermore, we can use HiveMQ, an MQTT proxy that can be deployed in a distributed multi-node manner, to improve the horizontal expansion capability of the MQTT broker service.

[0039] In one embodiment, the Nginx container image is used as a static file server, and based on the React-Admin front-end framework, a front-end part for managing model files in the adapter node is developed to achieve front-end and back-end separation. The model management front-end static file service is also located on the main node, and a front-end part of a background website that manages all model files running in the data acquisition adapter module is developed. The separation of the front-end and back-end can not only decouple the front-end user interface from the back-end data service, facilitate the team's task division and manpower arrangement, and enable the front-end and back-end of the website to be independently developed, tested, and launched, but also is a demonstration and embodiment of the microservice design concept, which is convenient for further promotion and implementation in more software development fields.

[0040] In one embodiment, a TCP reverse proxy service for MQTT protocol traffic is deployed on the K3s master node. The Ingress functional component provided by the K3s cluster is similar to a traditional reverse proxy, which can divert and forward external network traffic, but is more suitable for HTTP protocol traffic. MQTT's TCP data traffic cannot be simply forwarded using Ingress. This TCP reverse proxy can be easily configured through the traditional Nginx stream module.

[0041] In one embodiment, in order to save the configuration of the model file required by the NC-Link data acquisition adapter, a MongoDB non-relational document database capable of persisting data is deployed in the cluster, which can facilitate the flexible saving of model files defined in JSON text format. The database is deployed in the form of a container image, which facilitates horizontal expansion in the K3s cluster to cope with higher access traffic.

[0042] In one embodiment, a microservice module for model configuration and distribution is designed. The microservice module is used to read the model configuration and model definition information in the database, and provides an interface facing the cluster intranet to facilitate the NC-Link adapter call that needs to obtain or update the data model. In this way, a bridge and channel between the model database and the model use terminal are established. Because its function is closely related to the database, in order to reduce the delay and overhead of network transmission, the module and the model database can be integrated into the same node.

[0043] In one embodiment, a REST service for backend management is designed, which corresponds to the frontend service module and customizes various standard API interfaces for the frontend. Through this frontend-backend separation design, various required standard API interfaces can be flexibly customized for the frontend at the backend of the website, and operations such as adding, deleting, modifying, and checking target records in the database can be performed. While ensuring the basic functions of the website, necessary user authority management, data security, and concurrent security can be achieved through middleware, routing design, data access mutex locks, and other methods. Enhanced services.

[0044] In one embodiment, the adapter node includes an adapter module, which is used to perform protocol parsing, data filtering, and NC-Link format conversion operations on native data formats or other existing industrial Internet data collection protocols, such as OPC-UA, MTConnect, etc., so that data can be collected from a variety of industrial Internet edge device data sources. Integrating as many protocol adapter interfaces as possible can solve the complexity problems caused by different communication interfaces and protocols in the data collection nodes at the edge of the network, standardize the format of the transmitted data as early as possible, and avoid these complexities from being passed along the data transmission chain to the downstream modules, causing more complexity to be solved.

[0045] For example, Figure 5 As shown, for the K3s master service node, it can include the K3s cluster control plane, Rancher cluster background management, front-end static file service and Nginx MQTT reverse proxy; for the K3s working node, it can include MongoDB database, NC-Link model configuration service, model management backend REST service and MQTT broker service.

[0046] After all microservice modules are developed, Docker can be used to build the application, and finally generate an optimized, small-sized container image that supports multiple CPU architectures. Based on these standard images, you can continue to use the various components provided by the Kubernetes protocol framework to write applicable YAML definition scripts for various applications, and finally accurately and reliably implement batch deployment of K3s clusters.

[0047] At the same time, based on the microservice container development model, the containerized software design concept of "develop once, run in multiple places, and expand on demand" can be realized, effectively avoiding repeated development problems and improving work efficiency. At the same time, general modules such as the NC-Link adapter are most suitable for packaging and distribution in the form of container images that support multiple CPU architectures. On heterogeneous devices, batch deployment through Kubernetes clusters is convenient and fast, and it is not easy to have compatibility problems that are difficult to track and troubleshoot in conventional development. Through various data acquisition microservice modules, industrial Internet standard data communication protocols, such as NC-Link, can be applied to standardize the data components and structural forms that are ultimately transmitted to the edge computing system, shield the different details of the underlying data source, and provide convenience and conditions for subsequent further data analysis and processing.

[0048] It is understandable that the overall architecture of the Industrial Internet K3s containerized platform is as follows Figure 6 As shown in the figure, for external request access, the NC-Link agent can send collection tasks, distribute model files, and request data through gRPC or MQTT, and the NC-Link adapter performs corresponding data collection. Among them, the K3s main service node can control and manage the agent and adapter.

[0049] Based on the embodiments provided by the present invention, the problem of industrial data acquisition systems being difficult to deploy and maintain in non-containerized platforms can be solved, and computing resources can be flexibly expanded and allocated within the system's software and hardware resource capabilities to dynamically provide intelligent data acquisition services on demand. On the K3s platform, data acquisition resources can be simply expanded, and computing resources can be dynamically provided based on demand, which is more secure and reliable.

[0050] In one embodiment of the present invention, there is also provided an industrial data collection method, the method comprising:

[0051] Release data collection tasks based on Ansible control nodes and manage K3s clusters;

[0052] Manage, monitor, maintain and schedule K3s working nodes through K3s master service nodes, and perform specific data collection and processing tasks through K3s working nodes;

[0053] Industrial equipment data is acquired based on the adapter node and processed through the agent node.

[0054] The K3s working nodes include: a data collection service agent node and a data collection service adapter node.

[0055] The agent node includes: a model file database, a model file distribution module, a model file management website backend module and an MQTT agent module.

[0056] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0057] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An industrial data acquisition platform, characterized in that: At least: Ansible control node, used to publish industrial data collection tasks, manage all nodes in the K3s cluster based on the SSH protocol, and implement batch system configuration, program deployment, and command execution; K3s master service node, used to manage, monitor, maintain and schedule K3s working nodes; The K3s master service node includes: Rancher cluster runs on the K3s master service node and is a web application used to manage the entire K3s cluster; Among them, the Rancher cluster is deployed in the K3s cluster through Helm for K3s cluster management, application deployment, cluster operation and maintenance, and cluster monitoring; K3s working node, used to perform corresponding data collection and data processing tasks; The K3s working node includes a data collection agent node and an adapter node, wherein the agent node is deployed on an edge network and the adapter node is deployed on a data collection device; The proxy node includes a file management module and a file distribution module. The file management module realizes data transmission between the front-end and back-end of the website and the database through the REST interface, and the file distribution module performs database reading and model file transmission through the gRPC communication protocol. Wherein, the adapter node comprises: The adapter module is used to perform protocol parsing, data filtering, and NC-Link format conversion operations on native data formats or other existing industrial Internet data acquisition protocols.

2. The platform according to claim 1, characterized in that The K3s master service node contains the configuration information of the K3s cluster. The K3s master service node sets corresponding Kubernetes labels and stains according to the different processing tasks of the working nodes.

3. The platform according to claim 1, characterized in that: The agent node comprises: MQTT message queue middleware is used to decouple the adapter and the proxy, a certain number of proxies are deployed, and load balancing is performed among the proxy nodes.

4. The platform according to claim 1, characterized in that: The K3s master service node includes: Use the Nginx container image as a static file server and develop the front-end part for managing model files in the adapter node based on the React-Admin front-end framework to achieve front-end and back-end separation.

5. The platform according to claim 1, characterized in that: The adapter node comprises: The microservice module is used to read the model configuration and model definition information in the database, and provide an interface for the K3s cluster intranet so that the adapter can call the corresponding data model.

6. The platform according to claim 1, characterized in that: The data transmission between the front-end and back-end of the website and the database through the REST interface includes: Design REST services for backend management, which correspond to frontend service modules and customize standard API interfaces for various requirements for the frontend.

7. An industrial data collection method using the industrial data collection platform according to any one of claims 1 to 6, characterized in that: include: Release industrial data collection tasks based on Ansible control nodes and monitor and manage K3s clusters; Manage, monitor, maintain and schedule K3s working nodes through K3s master service nodes, and execute corresponding data collection and processing tasks through K3s working nodes; Industrial equipment data is acquired based on the adapter node and processed through the agent node.

Images