Faas-based Internet of Things cloud native device access system and method

Abstract

The invention discloses an internet of things cloud native device access system and method based on faas. The system comprises seven parts including a k8s platform, an FaaS engine, a user function, a message queue, an internet of things service, a mirror image warehouse and a user interaction interface. The system is high in resource utilization rate. And on the basis of automatic capacity expansion and contraction of FaaS, the system can automatically recover the function container when the flow is relatively low, so that the occupation of system resources is reduced. Meanwhile, the K8s provides a detailed resource management function, and distribution and use of resources such as a CPU and a memory can be finely controlled. And equipment protocol analysis is universal. The system supports more languages, including common python, java, c # and the like, and is not limited by implementation languages of other components of the Internet of Things. And service isolation. Based on the namespace of k8s, different businesses and different devices can use different namespaces and different containers, and the services do not influence each other.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a FaaS-based Internet of Things cloud native device access system and method. Background Art

[0002] IoT systems often need to connect to thousands or even more devices, and there are many types of devices with different access methods. Usually, the system will build a cluster by adding more servers to carry more concurrent requests; connect to different data transmission protocols by configuring gateways of different protocols by users; and support a variety of business data protocols by users uploading custom jar files or executing custom scripts.

[0003] However, this implementation method has some problems: low system utilization, device data reporting is often periodic, the server load is low during the reporting interval, and there is a waste of resources. Business access is difficult, and the new access to a device often requires developers to understand the access method defined by the IoT system, and usually also needs to match the development language of the IoT system. The operation and maintenance cost is high, and there is a lack of isolation measures for different businesses and different devices. Parsing anomalies of a certain device may cause anomalies of all services.

[0004] The information disclosed in this background technology section is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as acknowledging or suggesting in any form that the information constitutes the prior art already known to those skilled in the art. Summary of the invention

[0005] One of the purposes of the present invention is to provide an IoT cloud native device access system based on FaaS, which ensures high availability and scalability of the system through an automated deployment and expansion mechanism. Another purpose of the present invention is to provide an IoT cloud native device access method based on FaaS.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: The present invention provides a FaaS-based IoT cloud native device access system, the system comprising: k8s platform: used to manage service containers, including container start, stop, delete, and scale; Used to manage container services, including service discovery, load balancing, and network communication; Used to ensure high availability and stability of the system and monitor container operation, including automatically restarting failed containers, replacing killed containers, and destroying unhealthy containers; User interaction interface: used to enter the user's function code and manage IoT device information; FaaS engine: builds corresponding running containers according to the code entered by the user; it is used to access the network request of the device and route the request to the specific user container according to the port and request path; it is used to set service groups according to user settings; it is used to automatically scale the container according to the user settings and the actual network traffic size; User function: a function instance started by the FaaS component, which is used to parse the raw data reported by the device into the common data format of the IoT platform and then send it to the message queue; Message queue: used to receive parsed device reported data and deliver it to the message listener; IoT service: monitors the data reported by the device and writes it into the database; Image repository: used to manage container images, including template images of user function codes and other system images.

[0007] Furthermore, the k8s platform deploys a k8s cluster through the kubeadm tool. A cluster contains multiple nodes, and multiple nodes provide load balancing and fault recovery capabilities. The k8s platform supports the creation of multiple namespaces to isolate resources of different businesses and ensure the security and manageability of the system.

[0008] Furthermore, the system selects openfaas, kubeless or FaaS products provided by cloud service providers to deploy the FaaS engine; the FaaS engine includes function storage, compilation and construction, gateway, monitoring and alarm, controller, and data communication components; function storage is used to save user function code and provide code version management; compilation and construction is used to compile user function code, and build container images after compilation; the gateway is used to receive external requests and route them to corresponding functions; monitoring and alarm are used to monitor the running status of the function, including the number of requests and running load, and at the same time determine whether to trigger alarm information according to the rules configured by the user, and push it to the user; the controller controls the startup and expansion and contraction of the user function based on the real-time data monitored; the data communication component is used to communicate with other components.

[0009] Furthermore, the user function is a container started by the FaaS engine; when the device requests for the first time, the FaaS engine will start the corresponding image to create a container according to the configuration of the function and execute the startup script of the image; after the container is created, FaaS will use the device request raw data to call the data parsing code in the container; the data parsing code generates parsed device data according to the user logic and delivers it to the message queue; the FaaS engine dynamically creates and destroys user functions.

[0010] A method for accessing an IoT cloud native device based on FaaS, the method comprising: S1. Write function code: The user selects a suitable programming language based on the business logic, completes the function code writing and saves it; S2. Configure function running parameters: The user configures the execution parameters of the function, including the minimum and maximum number of instances and routing rules; S3, function compilation and container construction: The FaaS component compiles the code based on the code submitted by the user and builds the container image at the same time; S4, container startup: When the system receives a device request, it finds the corresponding user function based on the port and routing rules. If there is no running instance of the function at this time, the system immediately starts the container and processes the request. Based on the FaaS configuration, the system dynamically scales the user function according to the actual network traffic. S5. Receive and store device messages: The user function sends the parsed data to the message queue, the IoT writing module consumes the message, and writes the device data into the database.

[0011] By adopting the above technical solution, the present invention has the following beneficial effects: 1. High utilization of system resources. Based on the automatic expansion and contraction of FaaS, the system will automatically recycle function containers when the traffic is low, reducing the occupation of system resources. At the same time, K8s provides detailed resource management functions, which can finely control the allocation and use of resources such as CPU and memory.

[0012] 2. The device protocol parsing is universal. The system supports more languages, including commonly used python, java, c#, etc., and is not limited to the implementation language of other IoT components.

[0013] 3. Business isolation. Based on the k8s namespace, different businesses and different devices can use different namespaces and containers, and services will not affect each other.

[0014] 4. Business monitoring. Both FaaS and k8s have mature monitoring systems, and the system can easily access monitoring functions.

[0015] 5. Automated operation and maintenance. Based on K8s container management, the system can be deployed and rolled back more conveniently, reducing the risk of human error. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A structural diagram of the FaaS-based IoT cloud native device access system provided in an embodiment of the present invention; Figure 2 A flowchart of a FaaS-based IoT cloud native device access method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0020] Combination Figure 1 As shown, this embodiment provides an IoT device access system based on FaaS cloud native, including seven components: k8s platform, FaaS engine, user function, message queue, IoT service, image warehouse, and user interaction interface. Figure 1 .

[0021] The K8s platform is the foundation of this system and is responsible for managing and orchestrating all containerized applications. It ensures high availability and scalability of the system through automated deployment and expansion mechanisms. A k8s cluster is usually deployed through tools such as kubeadm. A cluster contains multiple nodes, which can provide load balancing and fault recovery capabilities. K8s supports the creation of multiple namespaces to isolate resources for different businesses and ensure the security and manageability of the system.

[0022] The FaaS engine provides users with an efficient and flexible way to run custom functions in response to events or requests. It has an automatic expansion function that can dynamically adjust the number of function instances according to the load to ensure high performance and high availability. You can usually choose openfaas, kubeless or FaaS products provided by cloud service providers to deploy the FaaS engine. The FaaS engine usually includes components such as function storage, compilation and construction, gateway, monitoring and alarm, controller, and data communication. Function storage is used to save user function code and provide code version management. The compilation and construction component is used to compile the user's function code and build the container image after compilation. The gateway is used to receive external requests and route them to the corresponding function. Monitoring and alarm are used to monitor the running status of the function, including the number of requests, running load, etc. At the same time, according to the rules configured by the user, it determines whether to trigger the alarm information and push it to the user. The controller controls the startup and expansion and contraction of the user function based on the real-time data monitored. The data communication component is used to communicate with other components.

[0023] The user function is a container started by the FaaS engine. When the device requests for the first time, the FaaS engine will start the corresponding image to create a container based on the function configuration and execute the image startup script. After the container is created, FaaS will use the device request raw data to call the data parsing code in the container. The data parsing code generates parsed device data based on the user logic and delivers it to the message queue. User functions can have multiple instances, and the FaaS engine will dynamically create and destroy user functions.

[0024] Message queues are used to asynchronously transmit device data to IoT services, usually using middleware such as rabbitMQ, rocketMQ, and kafka. Message queues receive messages generated by user functions and write them to disk. IoT services, as consumers, read messages from message queues. Message queues can reduce system coupling and act as buffers during peak traffic periods to avoid system overload.

[0025] The IoT service provides functions such as device writing and reading. In this invention, the main focus is on the writing of device data. After the IoT service reads the device data message, it saves the data persistently in the database.

[0026] The image warehouse is used to store images, which mainly include basic images and user function images in the present invention. For each commonly used programming language, including python, java, c#, golang, php, etc., the system provides a corresponding basic image. The basic image includes the system environment, the system library that the programming language depends on, and the parsing code template. Adding user parsing code to the basic image can form a user function image, which is used to start the user function container.

[0027] The user interaction interface is used to provide an interactive interface for user interaction. The user interaction interface includes code editor, container building, function configuration, and data sending components. The code editor is used to enter the user's function code and provides functions such as code highlighting, automatic completion, and code formatting; the container building provides an interface for users to select function code and code version, compile code, and build containers; the function configuration is used to configure the execution parameters of the user function, including request routing rules, basic images, minimum and maximum number of instances, etc.; the data transmission component is used to communicate data with the FaaS engine.

[0028] Combination Figure 2 As shown, this embodiment also provides a FaaS cloud-native IoT device access method, including five steps: writing function code, compiling function and building container, configuring function running parameters, starting container, receiving device messages and storing them, as shown in FIG. Figure 2 .

[0029] Step 1: Write the function code. According to the business logic, the user enters the parsed function code in the code editor of the user interface. The data transmission component of the user interface sends the code to the FaaS engine, and the function storage component of the FaaS engine saves the function code. The user can modify the code in the code editor and save multiple versions of the function code.

[0030] Step 2: Function compilation and container construction. In the container construction interface of the user interface, the user selects the function code, code version, and base image, and chooses to start editing and building. The data transmission component of the user interface notifies the FaaS engine, and the compilation and construction component of the FaaS engine compiles the corresponding user function code. After compilation, the compilation and construction component builds the user function image based on the selected base image and compiled code. Afterwards, the data transmission component of the FaaS engine pushes the user function image to the image warehouse for storage.

[0031] Step 3: Configure function running parameters. The user enters the function execution parameters in the function configuration interface of the user interface. The data transmission component of the user interface sends the parameters to the FaaS engine, and the gateway of the FaaS engine sets the request routing rules based on the configured parameters.

[0032] Step 4: Start the container. When the device data is reported to the gateway of the FaaS engine for the first time, the gateway finds the corresponding user function according to the routing rules. During the first request, since there is no running user function, the FaaS engine controller will start a user function instance and pass the device's original data to the user function instance. Subsequent requests will be directly routed to the user function instance. At the same time, the FaaS monitoring and alarm component will monitor the device's request volume. When the request volume is too large, the FaaS controller will increase the user instance; when the request volume is too small within a certain period of time, the FaaS controller will reduce the user instance.

[0033] Step 5: Receive and store device messages. After receiving the original device data, the user function parses the device data in a unified system format according to the user's business code and delivers it to the message queue. The IoT service receives the parsed device data from the message queue and saves the device data to the database.

[0034] Here is an example: There is a device that reports its location information via HTTP. The original message is as follows: { "x": 116.4, "y": 39.5, "z": 1.1, "speed": 13.5, "status": "online", ... } The user first enters the parsing function code in the user interface using Python: def handle(event, context): path = event.getUri() body = event.getBody() id = path.split(" / ")[-1] message = { "deviceId": int(id), "metrics": { "locationX": body["x"], "locationY": body["y"], "locationZ": body["z"], "speed": body["speed"], "status": body["status"] } } messageHandler.publish(message) return {"statusCode": 200, "body": {"success": True}} After the compilation is complete, enter the execution parameters: Network port: 10000 Network protocol: http Request path: / biz / device / Minimum number of instances: 1 Maximum number of instances: 10 Start the function and complete the device data access. The device requests the address http: / / domain:10000 / biz / device / 13. The FaaS engine will route the request to the user function. After executing the function, a system message is generated: [ { "deviceId": 13, "metrics": { "locationX": 116.4, "locationY": 39.5, "locationZ": 1.1, "speed": 13.5, "status": "online" } } ] After receiving the message, the IoT service writes the device data into the database.

[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A FaaS-based IoT cloud native device access system, characterized in that: The system comprises: k8s platform: used to manage service containers, including container start, stop, delete, and scale; Used to manage container services, including service discovery, load balancing, and network communication; Used to ensure high availability and stability of the system and monitor container operation, including automatically restarting failed containers, replacing killed containers, and destroying unhealthy containers; User interaction interface: used to enter the user's function code and manage IoT device information; FaaS engine: builds corresponding running containers according to the code entered by the user; routes network requests for access devices to specific user containers according to ports and request paths; sets service groups according to user settings; and automatically scales containers according to user settings and actual network traffic size; User function: a function instance started by the FaaS component, which is used to parse the raw data reported by the device into the common data format of the IoT platform and then send it to the message queue; Message queue: used to receive parsed device reported data and deliver it to the message listener; IoT service: monitors the data reported by the device and writes it into the database; Image repository: used to manage container images, including template images of user function codes and other system images.

2. The faas-based IoT cloud native device access system according to claim 1 is characterized in that: The k8s platform deploys a k8s cluster through the kubeadm tool. A cluster contains multiple nodes, and multiple nodes provide load balancing and fault recovery capabilities; The k8s platform supports the creation of multiple namespaces to isolate resources of different businesses and ensure the security and manageability of the system.

3. The faas-based IoT cloud native device access system according to claim 1 is characterized in that: The system selects openfaas, kubeless or FaaS products provided by cloud service providers to deploy the FaaS engine; the FaaS engine includes function storage, compilation and construction, gateway, monitoring and alarm, controller, and data communication components; Function storage is used to save user function codes and provide code version management; The compilation and construction is used to compile the user's function code and build the container image after the compilation is successful; the gateway is used to receive external requests and route them to the corresponding functions; the monitoring alarm is used to monitor the running status of the function, including the number of requests and the running load, and at the same time, determine whether to trigger the alarm information based on the rules configured by the user and push it to the user; the controller controls the startup and scaling of the user function based on the real-time monitoring data; the data communication component is used to communicate with other components.

4. The faas-based IoT cloud native device access system according to claim 1, characterized in that: The user function is a container started by the FaaS engine. When a device requests the function for the first time, the FaaS engine starts the corresponding image to create a container and executes the image startup script according to the function configuration. After the container is created, FaaS uses the device request raw data to call the data parsing code in the container. The data parsing code generates parsed device data based on user logic and delivers it to the message queue; the FaaS engine dynamically creates and destroys user functions.

5. A method for accessing an IoT cloud native device using the access system described in any one of claims 1 to 4, characterized in that: The method comprises the following steps: S1. Write function code: The user selects a suitable programming language based on the business logic, completes the function code writing and saves it; S2. Configure function running parameters: The user configures the execution parameters of the function, including the minimum and maximum number of instances and routing rules; S3, function compilation and container construction: The FaaS component compiles the code based on the code submitted by the user and builds the container image at the same time; S4, container startup: When the system receives a device request, it finds the corresponding user function based on the port and routing rules. If there is no running instance of the function at this time, the system immediately starts the container and processes the request. Based on the FaaS configuration, the system dynamically scales the user function according to the actual network traffic. S5. Receive and store device messages: The user function sends the parsed data to the message queue, the IoT writing module consumes the message, and writes the device data into the database.

Images