Internet of Things data processing method and system based on edge computing

Abstract

The invention relates to the technical field of computers, in particular to an Internet of Things data processing method and system based on edge computing, and the method comprises the steps: obtaining multi-sensor data of Internet of Things equipment, converging the multi-sensor data, compressing redundant information, and obtaining processed sensor data; according to a sensor data processing application program, deploying a containerized application on an edge device through Kubernetes; adjusting the Pod quantity of the containerized application according to the load condition of the edge equipment; obtaining an AI model trained by the cloud, carrying out lightweight processing on the AI model, and converting the AI model subjected to the lightweight processing into a format adaptive to the operation of the edge device; and packaging the converted AI model into a container, and deploying the container in a Kubernetes cluster, so that the sensor data is locally processed through the AI model and does not need to be transmitted to a cloud for processing. The data processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an Internet of Things data processing method and system based on edge computing. Background Art

[0002] With the rapid development of the Internet of Things (IoT), more and more devices are connected to the internet. These devices generate enormous amounts of data, posing unprecedented challenges to data processing and analysis. While traditional cloud computing models offer high performance, reliability, and scalability, they suffer from high latency and bandwidth consumption when processing IoT data. Consequently, edge computing, an emerging computing paradigm, has emerged, aiming to move computing and data processing capabilities from the cloud closer to the data source. This approach reduces data transmission latency, improves data processing efficiency, and reduces reliance on cloud computing centers.

[0003] Traditional edge computing devices have fixed functions and lack development capabilities, making it difficult to meet the diverse business needs of small and medium-sized enterprises. In addition, edge devices have limited computing capabilities and do not fully consider load balancing issues, resulting in low data processing efficiency. Summary of the Invention

[0004] Based on this, it is necessary to provide an IoT data processing method and system based on edge computing that can improve data processing efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides an IoT data processing method based on edge computing, the method comprising: Establishing a connection between the edge device and the IoT device, obtaining multi-sensor data of the IoT device, and aggregating the multi-sensor data and compressing redundant information to obtain processed sensor data; Deploying a containerized application on the edge device through Kubernetes based on the sensor data processing application, wherein the containerized application is encapsulated in a Pod, and the Pod is used to manage the containerized application; Adjust the number of Pods of the containerized application according to the load of the edge device to achieve dynamic expansion and contraction and resource management of the edge device; Obtaining an AI model trained in the cloud, performing lightweight processing on the AI ​​model, and converting the lightweight AI model into a format suitable for operation on the edge device; The converted AI model is packaged into a container and deployed in a Kubernetes cluster, so that the processed sensor data can be processed locally by the AI ​​model without being transmitted to the cloud for processing.

[0006] In one embodiment, adjusting the number of Pods of the containerized application according to the load of the edge device to achieve dynamic expansion and contraction and resource management of the edge device includes: The Bi-LSTM load prediction model is used to capture the long-term dependencies and short-term fluctuations of edge device loads, and the load is predicted based on these long-term dependencies and short-term fluctuations. Determine whether the actual load exceeds the predicted load; If so, the number of Pods of the containerized application is dynamically adjusted through an overload compensation algorithm to achieve dynamic expansion and contraction and resource management of the edge device.

[0007] In one embodiment, dynamically adjusting the number of Pods of the containerized application using an overload compensation algorithm includes: Obtain resource usage of the Pod, and dynamically adjust the number of Pods of the containerized application or the resource quota of each Pod based on the resource usage of the Pod through an overload compensation algorithm.

[0008] In one embodiment, obtaining the AI ​​model trained on the cloud, performing lightweight processing on the AI ​​model, and converting the lightweight AI model into a format suitable for operation on the edge device includes: Obtain the AI ​​model trained in the cloud and remove redundant connections and parameters of the AI ​​model through model pruning; By quantizing, the parameters of the AI ​​model are expressed in a low-precision form to achieve lightweight processing of the AI ​​model; The lightweight AI model is converted into a format suitable for the operation of the edge device.

[0009] In one embodiment, before obtaining the AI ​​model trained in the cloud and deleting redundant connections and parameters of the AI ​​model through model pruning, the method further includes: By searching the network structure, a search space is defined, where the search space represents the range of AI model structures that can be searched. Selecting a search strategy based on the task performed on the edge device and the search space to search for all possible candidate AI models; During the search process, the performance of the candidate AI models is evaluated, and based on the evaluation results, the search strategy is updated until the optimal AI model is selected.

[0010] In one embodiment, before establishing a connection between the edge device and the IoT device, the method further includes: Determine whether the IoT device uses the MQTT protocol; If not, the IoT device protocol is converted to the MQTT protocol through a protocol converter, enabling the IoT device to communicate with the edge device.

[0011] In one embodiment, before adjusting the number of Pods of the containerized application according to the load of the edge device, the process further includes: Based on the edge federation architecture, traffic patterns of user requests are collected through traffic analyzers; Calculating the optimal resource allocation plan through a central optimizer based on the traffic pattern of the user request and the user's geographic location information; According to the resource allocation scheme, the user request is redirected to the correct edge device through the scheduler.

[0012] In a second aspect, the present application also provides an IoT data processing device based on edge computing. The device includes: The sensor data processing module is used to establish a connection between the edge device and the IoT device, obtain multi-sensor data of the IoT device, and aggregate the multi-sensor data and compress redundant information to obtain processed sensor data; A container deployment module is used to process the application program based on the sensor data and deploy the containerized application on the edge device through Kubernetes. The containerized application is encapsulated in a Pod, and the Pod is used to manage the containerized application. A container dynamic adjustment module is used to adjust the number of Pods of the containerized application according to the load of the edge device to achieve dynamic expansion and contraction and resource management of the edge device; A model lightweight processing module is used to obtain the AI ​​model trained in the cloud, perform lightweight processing on the AI ​​model, and convert the lightweight AI model into a format suitable for operation on the edge device; The model deployment module is used to package the converted AI model into a container and deploy it in a Kubernetes cluster, so that the processed sensor data can be processed locally by the AI ​​model without being transmitted to the cloud for processing.

[0013] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed: Establishing a connection between the edge device and the IoT device, obtaining multi-sensor data of the IoT device, and aggregating the multi-sensor data and compressing redundant information to obtain processed sensor data; Deploying a containerized application on the edge device through Kubernetes based on the sensor data processing application, wherein the containerized application is encapsulated in a Pod, and the Pod is used to manage the containerized application; Adjust the number of Pods of the containerized application according to the load of the edge device to achieve dynamic expansion and contraction and resource management of the edge device; Obtaining an AI model trained in the cloud, performing lightweight processing on the AI ​​model, and converting the lightweight AI model into a format suitable for operation on the edge device; The converted AI model is packaged into a container and deployed in a Kubernetes cluster, so that the processed sensor data can be processed locally by the AI ​​model without being transmitted to the cloud for processing.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps: Establishing a connection between the edge device and the IoT device, obtaining multi-sensor data of the IoT device, and aggregating the multi-sensor data and compressing redundant information to obtain processed sensor data; Deploying a containerized application on the edge device through Kubernetes based on the sensor data processing application, wherein the containerized application is encapsulated in a Pod, and the Pod is used to manage the containerized application; Adjust the number of Pods of the containerized application according to the load of the edge device to achieve dynamic expansion and contraction and resource management of the edge device; Obtaining an AI model trained in the cloud, performing lightweight processing on the AI ​​model, and converting the lightweight AI model into a format suitable for operation on the edge device; The converted AI model is packaged into a container and deployed in a Kubernetes cluster, so that the processed sensor data can be processed locally by the AI ​​model without being transmitted to the cloud for processing.

[0015] In summary, this application has the following beneficial technical effects: Aggregating multi-sensor data and compressing redundant information on edge devices avoids the high latency problem of transmitting all data to the cloud in the traditional model; deploying containerized applications on edge devices based on Kubernetes, and adjusting the number of Pods of containerized applications according to the load of edge devices, and being able to adjust computing resources in real time according to the load; lightweight processing of AI models trained in the cloud and converting them into a format suitable for edge device operation, so that sensor data can be processed locally through the AI ​​model, realizing efficient local operation of the model and thus improving data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 1 is a flow chart of an IoT data processing method based on edge computing in one embodiment; Figure 2 Schematic diagram of a flow chart of an IoT data processing method based on edge computing in another embodiment; Figure 3 This is a structural block diagram of an IoT data processing device based on edge computing in one embodiment; Figure 4 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0017] The embodiments of the present invention provide an Internet of Things data processing method and system based on edge computing.

[0018] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0019] In the description of the embodiments disclosed herein, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to." The term "based on" should be understood as "based, at least in part, on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The terms "first," "second," etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0020] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the Internet of Things data processing method based on edge computing in an embodiment of the present invention includes: S100, establish a connection between the edge device and the IoT device, obtain multi-sensor data of the IoT device, aggregate the multi-sensor data and compress redundant information to obtain processed sensor data.

[0021] Specifically, edge devices, as an essential component of IoT systems, are responsible for communication and data exchange with IoT devices. IoT devices typically include various sensors (such as temperature, humidity, light, and pressure). These sensors transmit data to edge devices via wireless or wired means. Edge devices possess certain computing and storage capabilities, enabling preliminary processing and analysis of sensor data. Sensors in IoT devices can be of the same or heterogeneous type, each collecting different physical quantities or environmental parameters. The data collected by these sensors needs to be collected and processed uniformly to form a complete multi-sensor dataset. After the multi-sensor data is aggregated to the edge device, it needs to be preprocessed, including cleaning, denoising, and calibration. Then, compressed sensing technology is used to reduce the data volume.

[0022] In this embodiment, by aggregating and compressing data on edge devices, the amount of data transmitted can be reduced, thereby reducing network bandwidth consumption and reducing data transmission delay.

[0023] S200, processes applications based on sensor data and deploys containerized applications on edge devices through Kubernetes.

[0024] Among them, Kubernetes is a cluster management platform based on container technology, which is used to automatically deploy, expand and manage containerized applications. Containerized applications refer to packaging an application and all its required dependencies into a container to form an independent, portable operating environment. Containerized applications are encapsulated in Pods to ensure the isolation of their operating environment. Pods are the smallest scheduling and management units in Kubernetes. They encapsulate one or more containers together to share network, storage and other resources, and provide logical abstraction and management.

[0025] Specifically, a Kubernetes cluster is established to containerize the sensor data processing application so that the containerized application can be deployed on the edge device. Edge devices (such as physical machines or virtual machines) can run Pods as Node nodes of Kubernetes. Each edge node can host multiple Pods, supporting distributed deployment and load balancing. The lightweight nature of Pods makes them suitable for resource-constrained edge devices. For example, through the automatic expansion (HPA) and rolling update functions of Pods, edge devices can dynamically adjust the number of Pod replicas to cope with sudden traffic or device failures. When deploying containerized applications on edge devices, you need to create a Pod resource object and specify the container's image, startup command, environment variables and other configuration information. At the same time, you also need to configure a Service resource object for the Pod to achieve service discovery and load balancing between Pods.

[0026] S300 adjusts the number of Pods of containerized applications based on the load of edge devices to achieve dynamic scaling and resource management of edge devices.

[0027] Specifically, one of Kubernetes' core functions in edge computing is resource management. It can dynamically adjust resource allocation on edge nodes (edge ​​devices) based on application needs, ensuring that each application has access to sufficient computing and storage resources. In an edge computing environment, an active hybrid autoscaler is used to achieve dynamic scaling of edge devices. Unlike traditional autoscalers (such as HPA, which only adjusts the number of pods), this autoscaler can simultaneously modify the number of pods and the resource quota of each pod (such as CPU and memory limits) without restarting the pod. This achieves a "combination of vertical and horizontal scaling" to more efficiently utilize the heterogeneous resources of edge nodes. The autoscaler combines a bidirectional long short-term memory (Bi-LSTM) load prediction model with an overload compensation algorithm. It can adjust the number of pods and resource quotas in advance based on the predicted load, thereby improving the responsiveness and stability of the system.

[0028] In this embodiment, Kubernetes dynamically adjusts container allocation according to the resource status of edge nodes, which can optimize resource utilization to adapt to the real-time needs of applications, thereby improving the reliability and scalability of the system.

[0029] S400 obtains the AI ​​model trained in the cloud, performs lightweight processing on the AI ​​model, and converts the lightweight AI model into a format suitable for edge device operation.

[0030] Specifically, first, the AI ​​model needs to be trained on a server or in the cloud. Standard deep learning frameworks such as PyTorch or TensorFlow can be used during the training process. The cloud then sends the trained AI model to the edge device. To adapt to the limitations of the edge device, the edge device needs to perform lightweight processing on the AI ​​model. Lightweight models can flexibly adjust the model's operation mode according to the resource status of the edge device through dynamic adjustment strategies and task scheduling algorithms, thereby finding the best balance between performance and efficiency. Among them, lightweight processing includes model compression, pruning, quantization, etc., which can reduce the size and computational complexity of the model while maintaining its performance level. Secondly, the lightweight AI model is converted into a format suitable for edge device operation. In addition, the appropriate edge device hardware should be selected based on the size and computing requirements of the model. For example, for models with high computing resource requirements, an edge computing box equipped with a high-performance processor can be selected; for some simple tasks, an ordinary embedded processor may be sufficient.

[0031] In this embodiment, by lightweighting the AI ​​model, the computational complexity of the model can be significantly reduced, enabling it to run efficiently on resource-constrained edge devices, avoiding uploading sensitive data to the cloud, thereby improving data security and user privacy protection.

[0032] S500 packages the converted AI model into a container and deploys it in a Kubernetes cluster, so that the processed sensor data can be processed locally by the AI ​​model without being transmitted to the cloud for processing.

[0033] Specifically, the converted AI model is packaged into a container, and the containerized AI model is deployed in a Kubernetes cluster to be integrated with applications on edge devices to achieve model reasoning and result output. After deployment, the model needs to be tested and verified to ensure its performance and accuracy on the edge device. The test may include indicators such as the model's reasoning speed, storage space occupancy, and accuracy. In addition, the edge device adopts an anti-dismantling self-destruction mechanism. This mechanism is usually achieved by installing a pressure sensor or a similar anti-destruction sensor inside the device. When the device casing is forcibly disassembled, the sensor will detect the anomaly and trigger the self-destruction program, thereby erasing sensitive data stored in the device, such as keys or encryption information. This design can effectively prevent unauthorized access and data leakage.

[0034] In this embodiment, the AI ​​model is containerized and deployed in a Kubernetes cluster to enable local processing of sensor data, thereby reducing the need to transmit data to the cloud and improving the system's response speed.

[0035] In one embodiment, Figure 2As shown, S300 includes: S320 uses the Bi-LSTM load prediction model to capture the long-term dependencies and short-term fluctuations of edge device loads and predict the load based on these dependencies and short-term fluctuations. S340, determining whether the actual load exceeds the predicted load; If so, S360 dynamically adjusts the number of Pods of the containerized application through an overload compensation algorithm to achieve dynamic scaling and resource management of edge devices.

[0036] Specifically, an active hybrid autoscaler is used to achieve dynamic scaling of edge devices. The core of the active hybrid autoscaler is Bi-LSTM, a bidirectional long-short-term memory network capable of processing both past and future information. Compared to traditional unidirectional LSTMs, Bi-LSTM is used as a load prediction model. By simultaneously considering both forward and backward information in the time series, Bi-LSTM can more accurately capture long-term dependencies and short-term fluctuations in load, thereby predicting load. Furthermore, the active hybrid autoscaler introduces an overload compensation algorithm. When the actual load exceeds the predicted load, the overload compensation algorithm dynamically adjusts the number of pods and resource quotas to prevent system overload.

[0037] In this embodiment, the active hybrid auto-scaler achieves intelligent management of Pod resources in Kubernetes and edge computing environments by combining the Bi-LSTM load prediction model and the overload compensation algorithm. The scaler can not only predict load changes in advance, but also compensate when the prediction is insufficient, thereby ensuring high availability of the system and efficient resource utilization.

[0038] In one embodiment, dynamically adjusting the number of Pods of a containerized application through an overload compensation algorithm includes: Obtain the resource usage of the Pod and, based on the Pod's resource usage, dynamically adjust the number of Pods for the containerized application or the resource quota of each Pod through an overload compensation algorithm.

[0039] Specifically, the active hybrid autoscaler supports not only horizontal scaling (HPA) but also vertical scaling (VPA) to achieve dynamic adjustment of Pods. HPA automatically adjusts the number of Pods based on the Pod's resource usage (such as CPU and memory usage). For example, when the CPU usage of an application reaches 80%, HPA automatically increases the number of Pod replicas to cope with the increased load; when the load decreases, HPA reduces the number of replicas to conserve resources. VPA dynamically adjusts each Pod's resource quota (such as CPU and memory) based on the Pod's resource requirements. It can automatically adjust the Pod's resource allocation based on load changes to adapt to different load requirements. The main goal of VPA is to optimize resource utilization and prevent Pods from retaining too many resources, thereby better utilizing cluster resources.

[0040] In this embodiment, the two mechanisms of HPA and VPA work together to manage resources more accurately and improve the elasticity and resource utilization of the system.

[0041] In one embodiment, obtaining an AI model trained on the cloud, lightweighting the AI ​​model, and converting the lightweighted AI model into a format suitable for running on an edge device includes: Obtain the AI ​​model trained in the cloud, and delete redundant connections and parameters of the AI ​​model through model pruning; express the parameters of the AI ​​model in a low-precision form through quantization to achieve lightweight processing of the AI ​​model; convert the lightweight AI model into a format suitable for edge device operation.

[0042] Specifically, lightweight processing includes model pruning and quantization. Model pruning is a model compression technology that simplifies and accelerates the model by identifying and removing parameters and connections that have little impact on model performance during training. Model pruning can be performed at the structural level of the AI ​​model, such as deleting entire neurons, convolution kernels, layers, etc., and can also perform unstructured pruning, that is, pruning the absolute value of the weight values ​​in the model's weight matrix. Through pruning, the size and computational complexity of the model can be significantly reduced, making it more suitable for deployment on resource-constrained devices. Quantization is a technology that represents the parameters of the model in a low-precision form, thereby reducing storage space and computational overhead. For example, through 8-bit quantization, the model's inference speed can be increased by 2.5 times, while the Top-1 accuracy rate only decreases by 1.5%. Quantization technology not only reduces the storage requirements of the model, but also improves the model's operational efficiency.

[0043] In one embodiment, obtaining the AI ​​model trained in the cloud and, before deleting redundant connections and parameters of the AI ​​model through model pruning, further includes: Through network structure search, a search space is defined, which represents the range of AI model structures that can be searched. Based on the tasks performed on the edge device and the search space, a search strategy is selected to search for all possible candidate AI models. During the search process, the performance of the candidate AI models is evaluated, and based on the evaluation results, the search strategy is updated until the optimal AI model is selected.

[0044] Among them, Neural Architecture Search (NAS) is a systematic process that aims to find the optimal AI model structure by defining the search space, selecting the search strategy, evaluating candidate models, and continuously optimizing the search strategy based on the evaluation results.

[0045] Specifically, the search space is a core component of NAS. It defines all possible combinations of model structures and serves as the starting point for the search process. The size and complexity of the search space directly impact search efficiency and the performance of the resulting model. The search space typically includes the number of layers, the number of neurons per layer, connectivity, activation functions, and convolution methods. On edge devices, the search space typically limits the depth and width of the model to ensure that the model can run within limited computing resources. The search strategy is a key step in NAS, determining how candidate AI model structures are selected and evaluated from the search space. Common search strategies include reinforcement learning and evolutionary algorithms. On edge devices, the choice of search strategy also needs to consider computing resource constraints. For example, when using reinforcement learning or evolutionary algorithms, there is a trade-off between search accuracy and computational cost. Candidate AI models are evaluated for performance (e.g., accuracy, inference speed, memory usage, etc.), and the search strategy is updated based on the evaluation results to find the optimal AI model.

[0046] In this embodiment, by defining the search space, selecting the search strategy, evaluating the candidate models, and continuously optimizing the search strategy based on the evaluation results, the optimal AI model is finally selected. This can efficiently and automatically search for high-performance, low-computational-cost models, thereby better adapting to the resource limitations of edge devices.

[0047] In one embodiment, before establishing a connection between the edge device and the IoT device, the method further includes: Determine whether the IoT device uses the MQTT protocol; if not, convert the IoT device protocol to the MQTT protocol through a protocol converter, so that the IoT device can communicate with the edge device.

[0048] Among them, the MQTT protocol is a lightweight publish / subscribe messaging protocol that is widely used in communication between IoT devices. It has the characteristics of low overhead, high efficiency and easy deployment, and is very suitable for resource-constrained devices and network environments. In addition, the MQTT protocol is widely adopted in IoT systems and has become one of the de facto standard protocols.

[0049] Specifically, in the IoT system, devices may use a variety of different communication protocols, such as Modbus, OPC UA, CoAP, etc. In order to achieve unified communication between devices, edge devices usually have protocol conversion functions. Through the protocol converter, different protocols are converted into MQTT protocol, thereby realizing communication between IoT devices and edge devices.

[0050] In one embodiment, before adjusting the number of Pods of the containerized application based on the load of the edge device, the method further includes: Based on the edge federation architecture, the traffic patterns of user requests are collected through traffic analyzers. The optimal resource allocation plan is calculated through the central optimizer based on the traffic patterns of user requests and the user's geographic location information. Based on the resource allocation plan, the user request is redirected to the correct edge device through the scheduler.

[0051] Among them, the edge federation architecture is a distributed machine learning framework that combines edge computing and federated learning. It aims to achieve privacy protection, low latency and high scalability by training and aggregating models near the data source; the traffic pattern of user requests refers to the logical structure and behavioral characteristics followed by the requests initiated by users in a specific network environment. This traffic pattern can be independent of the physical infrastructure and focus on the logical attributes of the request, such as the source, target, frequency, time distribution, etc.

[0052] Specifically, the edge federation architecture typically includes components such as a traffic analyzer, a central optimizer, and a scheduler. The traffic analyzer collects the traffic patterns of user requests. This component is responsible for parsing user behavior data (such as request frequency, packet size, access time, etc.) and location information (such as IP address, base station location, etc.) based on the traffic patterns of user requests, providing input parameters for subsequent resource allocation. The central optimizer calculates the optimal resource allocation plan based on the data provided by the traffic analyzer and the user's geographic location information. This process involves multi-dimensional dynamic optimization, such as minimizing latency, maximizing resource utilization, or balancing loads. The scheduler dynamically redirects user requests to the most appropriate edge device based on the resource allocation plan calculated by the central optimizer, achieving efficient resource utilization.

[0053] In this embodiment, the edge federation architecture achieves efficient resource utilization and improved service quality through the synergy of traffic analysis, central optimization and dynamic scheduling.

[0054] In one embodiment, Figure 3 As shown, an IoT data processing device based on edge computing is provided, comprising: a sensor data processing module 10, a container deployment module 20, a container dynamic adjustment module 30, a model lightweight processing module 40 and a model deployment module 50, wherein: The sensor data processing module 10 is used to establish a connection between the edge device and the IoT device, obtain multi-sensor data of the IoT device, aggregate the multi-sensor data and compress redundant information to obtain processed sensor data; The container deployment module 20 is used to process the application program according to the sensor data and deploy the containerized application on the edge device through Kubernetes. The containerized application is encapsulated in a Pod, and the Pod is used to manage the containerized application. The container dynamic adjustment module 30 is used to adjust the number of Pods of the containerized application according to the load of the edge device to achieve dynamic expansion and contraction and resource management of the edge device; The model lightweight processing module 40 is used to obtain the AI ​​model trained in the cloud, perform lightweight processing on the AI ​​model, and convert the lightweight AI model into a format suitable for edge device operation; The model deployment module 50 is used to package the converted AI model into a container and deploy it in a Kubernetes cluster, so that the processed sensor data can be processed locally through the AI ​​model without being transmitted to the cloud for processing.

[0055] In one embodiment, the container dynamic adjustment module 30 is also used to capture the long-term dependencies and short-term fluctuations of the edge device load through the Bi-LSTM load prediction model, and predict the load based on the long-term dependencies and short-term fluctuations of the edge device load; determine whether the actual load exceeds the predicted load; if so, dynamically adjust the number of Pods of the containerized application through the overload compensation algorithm to achieve dynamic expansion and contraction and resource management of the edge device.

[0056] In one embodiment, the container dynamic adjustment module 30 is further used to obtain the resource usage of the Pod, and dynamically adjust the number of Pods of the containerized application or the resource quota of each Pod based on the resource usage of the Pod through an overload compensation algorithm.

[0057] In one embodiment, the model lightweight processing module 40 is also used to obtain the AI ​​model trained in the cloud, delete the redundant connections and parameters of the AI ​​model through model pruning; represent the parameters of the AI ​​model in a low-precision form through quantization to achieve lightweight processing of the AI ​​model; and convert the lightweight AI model into a format suitable for edge device operation.

[0058] In one embodiment, the IoT data processing device based on edge computing also includes an optimal model search module, which is used to define a search space through network structure search, where the search space represents the range of AI model structures that can be searched; based on the tasks performed on the edge device and the search space, a search strategy is selected to search for all possible candidate AI models; during the search process, the performance of the candidate AI models is evaluated, and based on the evaluation results, the search strategy is updated until the optimal AI model is selected.

[0059] In one embodiment, the IoT data processing device based on edge computing also includes a protocol conversion module for determining whether the IoT device uses the MQTT protocol; if not, the IoT device protocol is converted into the MQTT protocol through a protocol converter, so that the IoT device can communicate with the edge device.

[0060] In one embodiment, the IoT data processing device based on edge computing also includes a device orientation module, which is used to collect the traffic pattern of user requests through a traffic analyzer based on the edge federation architecture; calculate the optimal resource allocation plan through a central optimizer based on the traffic pattern of user requests combined with the user's geographic location information; and redirect the user request to the correct edge device through a scheduler based on the resource allocation plan.

[0061] Each module in the aforementioned edge computing-based IoT data processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device's memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0062] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store infrared image data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for processing Internet of Things data based on edge computing is implemented.

[0063] Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0064] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for processing Internet of Things data based on edge computing, characterized in that: include: Establishing a connection between the edge device and the IoT device, obtaining multi-sensor data of the IoT device, and aggregating the multi-sensor data and compressing redundant information to obtain processed sensor data; Deploying a containerized application on the edge device through Kubernetes based on the sensor data processing application, wherein the containerized application is encapsulated in a Pod, and the Pod is used to manage the containerized application; Adjust the number of Pods of the containerized application according to the load of the edge device to achieve dynamic expansion and contraction and resource management of the edge device; Obtaining an AI model trained in the cloud, performing lightweight processing on the AI ​​model, and converting the lightweight AI model into a format suitable for operation on the edge device; The converted AI model is packaged into a container and deployed in a Kubernetes cluster, so that the processed sensor data can be processed locally by the AI ​​model without being transmitted to the cloud for processing.

2. The method for processing IoT data based on edge computing according to claim 1, wherein: Adjusting the number of Pods of the containerized application according to the load of the edge device to achieve dynamic expansion and contraction and resource management of the edge device includes: The Bi-LSTM load prediction model is used to capture the long-term dependencies and short-term fluctuations of edge device loads, and the load is predicted based on these long-term dependencies and short-term fluctuations. Determine whether the actual load exceeds the predicted load; If so, the number of Pods of the containerized application is dynamically adjusted through an overload compensation algorithm to achieve dynamic expansion and contraction and resource management of the edge device.

3. The method for processing Internet of Things data based on edge computing according to claim 2, characterized in that: Dynamically adjusting the number of Pods of the containerized application through the overload compensation algorithm includes: Obtain resource usage of the Pod, and dynamically adjust the number of Pods of the containerized application or the resource quota of each Pod based on the resource usage of the Pod through an overload compensation algorithm.

4. The method for processing Internet of Things data based on edge computing according to claim 1, characterized in that: The obtaining of the AI ​​model trained on the cloud, lightweighting the AI ​​model, and converting the lightweighted AI model into a format adapted for operation on the edge device includes: Obtain the AI ​​model trained in the cloud and remove redundant connections and parameters of the AI ​​model through model pruning; By quantizing, the parameters of the AI ​​model are expressed in a low-precision form to achieve lightweight processing of the AI ​​model; The lightweight AI model is converted into a format suitable for the operation of the edge device.

5. The method for processing Internet of Things data based on edge computing according to claim 4, characterized in that: Before obtaining the AI ​​model trained in the cloud and deleting redundant connections and parameters of the AI ​​model through model pruning, the method further includes: By searching the network structure, a search space is defined, where the search space represents the range of AI model structures that can be searched. Selecting a search strategy based on the task performed on the edge device and the search space to search for all possible candidate AI models; During the search process, the performance of the candidate AI models is evaluated, and based on the evaluation results, the search strategy is updated until the optimal AI model is selected.

6. The method for processing Internet of Things data based on edge computing according to claim 1, characterized in that: Before establishing a connection between the edge device and the IoT device, the following steps are also included: Determine whether the IoT device uses the MQTT protocol; If not, the IoT device protocol is converted to the MQTT protocol through a protocol converter, enabling the IoT device to communicate with the edge device.

7. The method for processing Internet of Things data based on edge computing according to claim 1, characterized in that: Before adjusting the number of Pods of the containerized application according to the load of the edge device, the method further includes: Based on the edge federation architecture, traffic patterns of user requests are collected through traffic analyzers; Calculating the optimal resource allocation plan through a central optimizer based on the traffic pattern of the user request and the user's geographic location information; According to the resource allocation scheme, the user request is redirected to the correct edge device through the scheduler.

8. An IoT data processing device based on edge computing, characterized in that: include: The sensor data processing module is used to establish a connection between the edge device and the IoT device, obtain multi-sensor data of the IoT device, and aggregate the multi-sensor data and compress redundant information to obtain processed sensor data; A container deployment module is used to process the application program based on the sensor data and deploy the containerized application on the edge device through Kubernetes. The containerized application is encapsulated in a Pod, and the Pod is used to manage the containerized application. A container dynamic adjustment module is used to adjust the number of Pods of the containerized application according to the load of the edge device to achieve dynamic expansion and contraction and resource management of the edge device; A model lightweight processing module is used to obtain the AI ​​model trained in the cloud, perform lightweight processing on the AI ​​model, and convert the lightweight AI model into a format suitable for operation on the edge device; The model deployment module is used to package the converted AI model into a container and deploy it in a Kubernetes cluster, so that the processed sensor data can be processed locally by the AI ​​model without being transmitted to the cloud for processing.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images