Chip edge Internet of Things data scheduling method, device and system

Abstract

The invention relates to the technical field of Internet of Things data processing, and discloses a chip edge Internet of Things data scheduling method, device and system. The method comprises the steps that an edge chip receives original data from Internet of Things equipment, intermediate data are generated, and the intermediate data are stored in a first cache region of the edge chip; setting a coordination node, initializing an equipment table by the coordination node, and storing all task request information into a task table; the coordination node checks whether the number of idle main edge devices in any main edge device is larger than a first preset value or not based on the device table, if yes, the remaining tasks in the task table are distributed to the idle main edge devices, and if not, the remaining tasks are distributed to one of the idle main edge devices, the active main edge devices and the idle working edge devices; and transmitting the data to the target node by the edge device for distributing the task, unpacking and extracting the intermediate data, and generating result data. According to the method, the efficiency and the reliability of chip edge internet-of-things data scheduling are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet of Things data processing, and in particular to a chip edge Internet of Things data scheduling method, device and system. BACKGROUND

[0002] With the rapid development of Internet of Things and edge computing technology, the amount of data generated by Internet of Things devices is growing exponentially, and traditional cloud computing architecture has been difficult to meet the real-time and high efficiency requirements. Edge computing can effectively reduce data transmission delay and improve system response speed by pushing computing power to the network edge and close to the data source. However, the processing power and resources of edge devices are limited, and how to efficiently schedule and manage these resources is a key problem.

[0003] Similar prior art includes Chinese patent application No. CN114265791A, which discloses a data scheduling method, including the following steps: obtaining the data to be scheduled by the first controller; in response to the destination address of the data to be scheduled being within the virtual address provided by the second controller, obtaining the data of each data page in the data to be scheduled in turn and determining the offset in the first data block in each data page according to the PRP address of each data page; using the second controller to send a data processing request to the first controller, so that the first controller sends the corresponding size of data in the current data page to the second controller according to the size of the data to be processed corresponding to the data processing request; the second controller determines the page sequence number of the data page to be stored and the page offset in the data page to be stored according to the virtual address range corresponding to the destination address of the corresponding size of data and the offset of the first data block in the current data page; write the corresponding size of data into the corresponding position in the data page to be stored according to the page sequence number and the page offset. Chinese patent application No. CN116483536A discloses a data scheduling method, computing chip and electronic device, the method comprising: sequentially issuing data requests to a device memory through a first type of thread bundle, and storing the data to be processed corresponding to the data request returned by the device memory into a target area of a target storage; wherein the target storage is a storage of a thread block to which the first type of thread bundle belongs, and the target area is a region in the target storage corresponding to the data request; for any data request, when all first type of thread bundles of the thread block store the data to be processed corresponding to the data request into the target area of the target storage, a second type of thread bundle is used to take the data to be processed from the target area of the target storage to a vector register to process the data to be processed.

[0004] The deficiencies of the prior art mainly lie in complex interaction between multiple controllers and fine management of data pages, which causes great pressure on computing resources and storage resources of edge devices; multiple thread bundles need to be allocated on a computing chip to complete data request, storage and processing tasks. In the edge Internet of Things scenario, edge devices usually have limited computing resources, and such a high resource occupation mode may cause performance bottleneck of edge devices. In actual situations, the actual needs of chip edge Internet of Things data scheduling need to be met from multiple aspects such as data preprocessing, task allocation, resource management, data transmission and encryption. SUMMARY

[0005] The present application provides a chip edge Internet of Things data scheduling method, device and system for improving the efficiency and reliability of chip edge Internet of Things data scheduling.

[0006] In a first aspect, the present application provides a chip edge Internet of Things data scheduling method, which comprises: An edge chip receives raw data from an Internet of Things device, pre-processes the raw data to generate intermediate data, and stores the intermediate data in a first cache area of the edge chip; A coordination node is set, which initializes a device table, wherein the device table includes device types, device states and device running data of any edge device, collects task request information from any edge device, wherein the task request information includes task types, working time and task priorities, and stores all the task request information into a task table; The coordination node checks whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table, and if yes, allocates remaining tasks in the task table to the idle main edge devices, otherwise, allocates the remaining tasks to one of the idle main edge devices, active main edge devices and idle working edge devices; A target node is extracted from the device table, and the edge device allocating the task transmits data to the target node, the target node receives a data packet, unpacks and extracts the intermediate data, pre-processes the intermediate data based on the data type, and generates result data.

[0007] In combination with the first aspect, the allocation of the remaining tasks in the task table to the idle main edge devices comprises: If there is only a task to be allocated in the task table, the idle main edge devices are constructed into a cluster, and the task to be allocated is allocated to the cluster; if the number of the idle master edge devices is less than the number of the remaining tasks in the task table, distributing the remaining tasks to the idle master edge devices until the number of the remaining tasks is equal to the number of the idle master edge devices; if the number of the idle master edge devices is greater than or equal to the number of the remaining tasks, calculating performance scores of the idle master edge devices, sorting the remaining tasks in descending order of working time, sorting the idle master edge devices in descending order of the performance scores, and distributing the remaining tasks to the idle master edge devices.

[0008] With reference to the first aspect, the distributing the remaining tasks to one of the idle master edge devices, the active master edge device, and the idle working edge device comprises: checking whether the active master edge device exists in the device table, if the active master edge device exists, distributing the to-be-distributed task to the active master edge device with the shortest working time, otherwise, distributing the to-be-distributed task to the idle working edge device; if the number of the idle working edge devices is greater than the first preset value, calculating a first complexity of any to-be-distributed task, and calculating an average complexity corresponding to all the remaining tasks in the task table; if the first complexity is greater than the average complexity, selecting the idle working edge device with the highest device performance to distribute the to-be-distributed task, otherwise, selecting the idle working edge device with the lowest device performance to distribute the to-be-distributed task.

[0009] With reference to the first aspect, the calculating the first complexity of any to-be-distributed task comprises: acquiring the intermediate data corresponding to the to-be-distributed task in the first cache area based on the task type, and setting the intermediate data as task data; performing feature extraction on the task data to generate task data features, and constructing a deep learning processing model based on the task data features; acquiring a model architecture of the deep learning processing model, extracting model parameters based on the model architecture, and calculating the first complexity based on the model parameters.

[0010] With reference to the first aspect, the edge device distributing the task transmits data to a target node, comprising: The edge device evaluates a data amount of the data to be transmitted, if the data amount is greater than a second preset value, generates a symmetric key using a random number generator, encrypts the data to be transmitted using an AES algorithm to generate first encrypted data, generates a check code for the first encrypted data, encapsulates and transmits the symmetric key, the check code and the first encrypted data to the target node, otherwise, generates a pair of public key and private key using an RSA algorithm, encrypts the data to be transmitted using the public key to generate the first encrypted data, generates a new check code for the first encrypted data, and encapsulates and transmits the first encrypted data and the new check code to the target node.

[0011] In combination with the first aspect, the unpacking and extracting the intermediate data comprises: The target node unpacks the received data packet, extracts encrypted data and a check code, verifies the integrity of the encrypted data based on the check code, if the data verification fails, the target node requests the edge device to resend the data packet, otherwise, decrypts the encrypted data based on the key in the data packet to generate the intermediate data.

[0012] In combination with the first aspect, the extracting the target node in the device table comprises: Based on the device running data, node resource information is obtained in the device table, and based on the task priority, a job configuration file of a task request is obtained in the task table; Based on the node resource information and the job configuration file, it is judged whether the available resources of the edge device in the device table meet the resource requirements of the task request, if the available resources of at least one edge device meet the resource requirements, direct resource allocation is performed, if the available resources of any edge device do not meet the resource requirements, and after occupying the available resources of low task priority, the resource requirements are met, indirect resource allocation is performed; The edge device after resource allocation is set as the target node.

[0013] In combination with the first aspect, a first node with available resources matching the job configuration file is obtained in the device table, the to-be-processed task is scheduled to the first node, and the to-be-processed task is put into an execution queue; when indirect resource allocation is performed, a second node with low task priority is obtained in the device table based on the task priority, the second node is backed up with a low task priority running state, and the use resources of the second node are released, the task request corresponding to the low task priority is put into a waiting queue, the to-be-processed task is scheduled to the second node, and the to-be-processed task is put into an execution queue.

[0014] In a second aspect, the application provides a chip edge IoT data scheduling system, which comprises: a receiving module, configured to receive raw data from an IoT device by an edge chip, and to pre-process the raw data to generate intermediate data, and to store the intermediate data in a first cache area of the edge chip; an association module, configured to set a coordination node, and to initialize a device table, wherein the device table comprises a device type, a device state, and device running data of any edge device, and to collect task request information from any edge device, and to store all the task request information in a task table, wherein the task request information comprises a task type, a working time, and a task priority; a scheduling module, configured to check, by the coordination node, whether a number of idle master edge devices in any master edge device is greater than a first preset value based on the device table, and if yes, to distribute remaining tasks in the task table to the idle master edge devices, and if not, to distribute the remaining tasks to one of the idle master edge devices, active master edge devices, and idle working edge devices; a generation module, configured to extract a target node from the device table, and to transmit data from an edge device assigned with a task to the target node, and to unpack and extract the intermediate data after the target node receives a data packet, and to pre-process the intermediate data based on a data type to generate result data.

[0015] In a third aspect, the application further provides an apparatus, characterized in that it comprises: a memory, configured to store a computer program; a processor, configured to execute the computer program to implement the above-mentioned chip edge IoT data scheduling method.

[0016] In the technical scheme provided in the application, first, in the task allocation process, different allocation strategies are adopted according to different conditions to realize efficient allocation and processing of tasks. By constructing a cluster, the tasks are allocated to the cluster, the cooperative working ability of the cluster is fully utilized, and the efficiency of task processing is improved. In turn, the tasks are allocated to the idle master edge device, ensuring that the existing resources are fully utilized. By calculating the performance score of the computing device and the working time of the task, reasonable sorting and allocation are performed, and the task is allocated to the most suitable device, further improving the efficiency of task processing and the overall performance of the system. Then, by checking the working time of the active master edge device, the task is allocated to the device that can complete the current task the fastest, thereby reducing the waiting time of the task. According to the actual calculation requirements and resource consumption of the task, complexity evaluation is performed, so that the device is more reasonably selected during task allocation, improving the resource utilization efficiency and task processing speed of the system. Through the introduction of a deep learning processing model, complex task data can be better processed, and different types of Internet of Things task requirements can be adapted. Finally, in the data transmission process, different encryption methods are selected according to the data size of the data to be transmitted, the target node performs unpacking, data integrity verification and decryption operations on the received data packet, ensuring the security and reliability of the data, verifying the integrity of the data based on the check code to avoid incorrect or tampered data entering the subsequent processing flow, generating intermediate data that can be used for subsequent processing, ensuring the safe transmission and accurate processing of data in the Internet of Things system, and improving the overall security and reliability of the system.

[0017] The application can also accurately determine whether the available resources of the edge device meet the resource requirements of the task request by obtaining node resource information and job configuration files. According to different resource conditions, the method flexibly selects direct resource allocation or indirect resource allocation, ensuring that the task can be reasonably allocated resources, improving the efficiency and accuracy of task scheduling. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings.

[0019] Figure 1 An embodiment of the chip edge Internet of Things data scheduling method in the embodiment of the application is shown in the figure; Figure 2 An embodiment of the data transmission and security processing flow in the embodiment of the application is shown in the figure; Figure 3 An embodiment of the task priority processing flow in the embodiment of the application is shown in the figure; Figure 4 FIG. 1 is a schematic diagram of an embodiment of a chip edge IoT data scheduling system according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] The present application provides a chip edge IoT data scheduling method, device and system. The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" or "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] For ease of understanding, the specific process of the embodiments of the present application is described below. Please refer to FIG. 1 Figure 1 An embodiment of a chip edge IoT data scheduling method in the present application includes the following steps. Step S101, the edge chip receives raw data from the IoT device, and pre-processes the raw data to generate intermediate data, and stores the intermediate data in the first cache area of the edge chip.

[0022] It can be understood that the execution subject of the present application can be a chip edge IoT data scheduling device, and can also be a terminal or a server, and the specific execution subject is not limited herein. The embodiments of the present application take the server as the execution subject for example.

[0023] Specifically, the edge chip serves as a preliminary processing unit for IoT device data, and receives raw data from the IoT device. These raw data can contain various types of information, such as temperature, humidity, pressure, etc. data collected by sensors, not limited to image, audio, etc. data types. The edge chip pre-processes the raw data, and the purpose of pre-processing is to convert the raw data into intermediate data suitable for subsequent processing. For example, for image data, pre-processing can include cropping, scaling, normalization, etc. operations; for sensor data, pre-processing can include data cleaning, filtering, etc. operations to remove noise and invalid data. The first cache area is an internal storage area of the edge chip, used to temporarily store the pre-processed data.

[0024] In step S102, a coordination node is set, the coordination node initializes a device table, the device table includes a device type, a device state, and device running data of any edge device, the coordination node collects task request information from any edge device, the task request information includes a task type, a working time, and a task priority, and stores all the task request information into a task table.

[0025] Specifically, the coordination node refers to a device node for coordinating data, which plays a role of management and scheduling in the whole system and is responsible for initializing the device table. The device table is a table recording all the related information of edge devices, including device types (such as main edge devices, working edge devices, etc.), device states (such as idle, busy, etc.), and device running data (such as the remaining time of the current task, the performance index of the device, etc.). The coordination node collects task request information from any edge device, which reflects the related situation of the task to be executed by the edge device. The task type refers to the type of the task, such as data processing, model training, etc.; the working time indicates the time required to complete the task; and the task priority is used to indicate the importance and urgency of the task, so as to prioritize when assigning tasks. The coordination node stores all the collected task request information into the task table, which is used to record and manage all the tasks to be processed.

[0026] In step S103, the coordination node checks whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table, if yes, assigns the remaining tasks in the task table to the idle main edge devices, otherwise, assigns the remaining tasks to one of the idle main edge devices, active main edge devices, and idle working edge devices.

[0027] Specifically, the coordination node checks whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table. The main edge device is an edge device with higher computing power and management function, and the idle main edge device refers to the main edge device that has no task being executed at present. If the number of idle main edge devices is greater than the first preset value, it means that there are enough main edge devices to process the remaining tasks, and then the coordination node assigns the remaining tasks in the task table to the idle main edge devices. The resources of the main edge devices can be fully utilized, and the efficiency of task processing can be improved. If the number of idle main edge devices is not greater than the first preset value, it means that the resources of the main edge devices are relatively tight, and at this time the coordination node assigns the remaining tasks to one of the idle main edge devices, active main edge devices, and idle working edge devices. The active main edge device refers to the main edge device that is currently executing a task, and the idle working edge device refers to an edge device with lower computing power but can assist in completing the task. This assignment method can flexibly select appropriate devices to execute tasks according to the actual situation of different devices and the characteristics of tasks, and improve the overall performance of the system.

[0028] In step S104, the target node is extracted from the device table, the edge device assigned with the task transmits data to the target node, and after the target node receives the data packet, it unpacks and extracts intermediate data, pre-processes the intermediate data based on the data type, and generates result data.

[0029] Specifically, the target node is extracted from the device table, which is the device node selected to receive and process data. The edge device assigned with the task transmits data to the target node, and data transmission can be achieved through wired or wireless communication. After the target node receives the data packet, it unpacks and extracts intermediate data. Unpacking is to parse the received data packet according to a certain format and extract the intermediate data. Then, the target node pre-processes the intermediate data based on the data type and generates result data. For example, if the intermediate data is image data, the target node may perform further image analysis and processing to extract key features in the image and generate result data. Result data is the final data after processing.

[0030] In a specific embodiment, the remaining tasks in the task table are assigned to the idle master edge devices, including: (1) If there is only a task to be assigned in the task table, the idle master edge devices are constructed into a cluster, and the task to be assigned is assigned to the cluster.

[0031] (2) If the number of idle master edge devices is less than the number of remaining tasks in the task table, the remaining tasks are assigned to the idle master edge devices until the number of remaining tasks is equal to the number of idle master edge devices.

[0032] (3) If the number of idle master edge devices is greater than or equal to the number of remaining tasks, the performance scores of the idle master edge devices are calculated, the remaining tasks are sorted by working time from long to short, the idle master edge devices are sorted by performance score from high to low, and the remaining tasks are assigned to the idle master edge devices.

[0033] Specifically, the coordination node checks the task table, and if it finds that there is only a task to be assigned in the task table (i.e., there are no other assigned or processing tasks), the idle master edge devices are constructed into a cluster. The cluster is a collaborative working unit composed of multiple master edge devices, and through clustering, the resources of multiple devices can be integrated. The task to be assigned is assigned to the constructed cluster. The master edge devices in the cluster can work collaboratively to complete the task to be assigned.

[0034] The coordination node calculates the number of idle master edge devices and the number of remaining tasks in the task table. If the number of idle master edge devices is less than the number of remaining tasks, it means that the existing idle master edge devices are insufficient to allocate all the remaining tasks. The coordination node allocates the remaining tasks to the idle master edge devices one task at a time until the number of remaining tasks is equal to the number of idle master edge devices. In this way, it can be ensured that each idle master edge device can be allocated a task, making full use of existing idle resources, while avoiding the situation that too many tasks cannot be allocated.

[0035] The coordination node calculates the performance score of the idle master edge device. The performance score can be calculated according to various performance indicators of the device, such as CPU processing power, memory capacity, storage speed, etc. The higher the performance score, the better the overall performance of the device. The coordination node sorts the remaining tasks in descending order of working time, which refers to the time required to complete each task. Sorting tasks by working time can prioritize tasks that take longer to process. Sort the idle master edge devices in descending order of performance score, and prioritize allocating tasks to devices with better performance to make full use of high-performance devices and improve the speed and efficiency of task processing. Assign the sorted remaining tasks to the sorted idle master edge devices, and assign tasks to the most suitable device according to the working time of the task and the performance score of the device to achieve optimal allocation of task processing.

[0036] In a specific embodiment, assigning the remaining tasks to one of the idle master edge devices, active master edge devices, and idle working edge devices includes: (1) Check in the device table whether there are active master edge devices. If there are, assign the task to be allocated to the active master edge device with the shortest working time, otherwise, assign the task to be allocated to the idle working edge device.

[0037] (2) If the number of idle working edge devices is greater than a first preset value, calculate the first complexity of any task to be allocated, and calculate the average complexity corresponding to all remaining tasks in the task table.

[0038] (3) If the first complexity is greater than the average complexity, select the idle working edge device with the highest device performance to allocate the task to be allocated, otherwise, select the idle working edge device with the lowest device performance to allocate the task to be allocated.

[0039] Specifically, the active master edge device refers to the master edge device currently performing tasks, and the active master edge device refers to the master edge device currently performing tasks. Assigning the to-be-assigned task to the active master edge device with the shortest working time can minimize the task waiting time and improve the utilization of the device. If there is no active master edge device, the coordination node assigns the to-be-assigned task to the idle working edge device, which refers to the working edge device currently without tasks being executed.

[0040] The first complexity is based on the calculation requirement, data volume and other factors of the task to measure the calculation. The average complexity is the sum of the first complexity of the remaining tasks divided by the number of the remaining tasks.

[0041] If the first complexity is greater than the average complexity, it means that the to-be-assigned task is relatively complex and requires higher computing power to process. Therefore, the idle working edge device with the highest device performance is selected for distribution. If the first complexity is less than or equal to the average complexity, it means that the to-be-assigned task is relatively simple, and the idle working edge device with lower device performance can be selected for distribution to make full use of resources and avoid wasting the ability of high-performance devices.

[0042] In a specific embodiment, the first complexity of any to-be-assigned task is calculated, including: (1) Obtain the intermediate data corresponding to the to-be-assigned task in the first cache area based on the task type, and set it as task data.

[0043] (2) Feature extraction is performed on the task data to generate task data features, and a deep learning processing model is constructed based on the task data features.

[0044] (3) Obtain the model architecture of the deep learning processing model, extract the model parameters based on the model architecture, and calculate the first complexity based on the model parameters.

[0045] Specifically, the task data refers to the intermediate data involved in executing the to-be-assigned task, different task types correspond to different task data, and different task data requires different processing methods.

[0046] Feature extraction is a process of extracting key information that can reflect the characteristics of data from task data. The purpose of feature extraction is to convert raw data into a more easily processed and analyzed form, reducing the dimensionality and complexity of data. For example, edge features and texture features in image data, or statistical features in sensor data. The deep learning processing model is a model based on deep neural networks, which can further process and analyze task data. The construction of the model can select appropriate network architecture according to the task type and data features, for example, convolutional neural network (CNN) for image processing, recurrent neural network (RNN) for time series data processing, etc.

[0047] The model architecture refers to the structure of the deep learning model, including the number of network layers, the number of neurons in each layer, the activation function, and other information. The model parameters are the weights and biases used for calculation and prediction in the deep learning model, which determine the performance and complexity of the model. The first complexity can be calculated based on the number of model parameters, the number of network layers, the computational complexity of each layer, and other factors. For example, the first complexity can be proportional to the number of model parameters, the more parameters, the higher the complexity of the model; it can also consider the computational complexity of each layer, such as the complexity of the convolutional layer, which is related to factors such as the size of the convolution kernel and the size of the feature map. By calculating the complexity indicators related to model parameters and architecture, the first complexity of the task to be allocated can be obtained.

[0048] In a specific embodiment, the edge device that allocates the task transmits data to the target node, including: The edge device evaluates the data volume of the data to be transmitted. If the data volume is greater than a second preset value, a symmetric key is generated using a random number generator, the data to be transmitted is encrypted using the AES algorithm to generate first encrypted data, a check code is generated for the first encrypted data, and the symmetric key, the check code, and the first encrypted data are encapsulated and transmitted to the target node. Otherwise, a pair of public and private keys is generated using the RSA algorithm, the data to be transmitted is encrypted using the public key to generate first encrypted data, a new check code is generated for the first encrypted data, and the first encrypted data and the new check code are encapsulated and transmitted to the target node.

[0049] Specifically, Figure 2The flowchart is for data transmission and security processing. The data volume refers to the size of the data to be transmitted, usually measured in bytes (Byte). If the data volume of the data to be transmitted is greater than a second preset value, the edge device uses a random number generator to generate a symmetric key, which is a kind of encryption key used for data encryption and decryption, and its characteristic is that the same key is used for encryption and decryption. The edge device uses the AES (Advanced Encryption Standard) algorithm to encrypt the data to be transmitted to generate the first encrypted data. AES algorithm is a kind of symmetric encryption algorithm, which has high encryption efficiency and security, and is suitable for encrypting a large amount of data. The first encrypted data generates a check code. The check code is a kind of code used to detect the integrity and accuracy of the data, which can detect whether the data has been error during transmission. The symmetric key, check code and first encrypted data are encapsulated and transmitted to the target node. Encapsulation refers to combining multiple data elements into a data packet for transmission. If the data volume of the data to be transmitted is less than or equal to the second preset value, the edge device uses the RSA algorithm to generate a pair of public and private keys. RSA algorithm is a kind of asymmetric encryption algorithm, its characteristic is that different keys are used for encryption and decryption, public key for encryption and private key for decryption. The edge device uses the public key to encrypt the data to be transmitted to generate the first encrypted data, and generates a new check code for the first encrypted data.

[0050] In a specific embodiment, the intermediate data is unpacked and extracted, including: The target node unpacks the received data packet, extracts the encrypted data and the check code, verifies the integrity of the encrypted data based on the check code, and if the data verification fails, the target node requests the edge device to resend the data packet, otherwise, the encrypted data is decrypted based on the key in the data packet to generate the intermediate data.

[0051] Specifically, unpacking refers to disassembling the encapsulated data packet and extracting the individual data elements it contains, such as encrypted data, checksum, and key. The target node extracts the encrypted data and checksum from the unpacked data. The checksum is an important basis for verifying whether the data is complete and has not been tampered with during transmission. The verification process is usually completed through a certain verification algorithm (such as CRC verification, hash verification, etc.). Specifically, the target node will use the same verification algorithm to recalculate the checksum for the encrypted data, and then compare the calculated checksum with the checksum extracted from the data packet. If the comparison result is inconsistent, i.e., the data verification fails, it means that the encrypted data may have been incorrect or tampered with during transmission. At this time, the target node will request the edge device to resend the data packet to ensure that the received data is complete and accurate. If the data verification is successful, it means that the encrypted data is complete and has not been tampered with. The target node will then extract the key from the data packet. The key may be a symmetric key (such as the key used in the AES algorithm), or a private key in asymmetric encryption (such as the private key used in the RSA algorithm). The decryption process is the inverse of the encryption process, and through decryption, the original intermediate data, i.e., the data preprocessed by the edge device, can be restored.

[0052] In a specific embodiment, the target node is extracted from the device table, including: (1) Obtain node resource information in the device table based on device running data, and obtain job configuration files of task requests in the task table based on task priority.

[0053] (2) Determine whether the available resources of the edge devices in the device table meet the resource requirements of the task requests based on the node resource information and the job configuration files. If the available resources of at least one edge device meet the resource requirements, direct resource allocation is performed. If the available resources of any edge device do not meet the resource requirements, and the available resources after occupying low-priority tasks meet the resource requirements, indirect resource allocation is performed.

[0054] (3) Set the edge devices after resource allocation as the target node.

[0055] Specifically, Figure 3 The task priority processing flowchart. The device running data reflects the current running state of the edge device, including the load condition of the device, available resources, etc. The node resource information is the resource condition of each edge device recorded in the device table, such as CPU usage, memory remaining, storage space, etc. The job configuration files of task requests are obtained in the task table based on task priority. The task priority determines the urgency and importance of the task, and the high-priority task will be allocated resources first. The job configuration file contains detailed information such as the type and quantity of resources required by the task, such as the number of CPU cores, memory size, etc.

[0056] The required resources in the job configuration file need to be compared with the actually available resources in the node resource information. If the available resources of at least one edge device meet the resource requirements of the task request, direct resource allocation is performed. Direct resource allocation refers to directly assigning the task to the edge device that meets the resource requirements. This method is simple and direct, and is suitable for the case of sufficient resources. If the available resources of any edge device do not meet the resource requirements of the task request, but can meet the resource requirements by pre-empting the available resources of a low-priority task, indirect resource allocation is performed. Indirect resource allocation refers to adjusting the existing resource allocation, such as suspending or terminating a low-priority task to release resources and then assigning the current task. This method is suitable for the case of resource shortage.

[0057] The edge device after resource allocation is set as a target node. The target node is the device that finally receives and processes the task, which is determined after resource allocation and is responsible for executing the task assigned to it.

[0058] In a specific embodiment, when direct resource allocation is performed, the first node with available resources matching the job configuration file is obtained in the device table, the task to be processed is scheduled to the first node, and the task to be processed is placed in the execution queue. When indirect resource allocation is performed, the second node with a low task priority is obtained in the device table based on the task priority, the running state of the second node with the low task priority is backed up, the use resources of the second node are released, the task request corresponding to the low task priority is placed in the waiting queue, the task to be processed is scheduled to the second node, and the task to be processed is placed in the execution queue.

[0059] Specifically, the first node with available resources matching the job configuration file is found in the device table. Here, “matching” means that the available resources of the first node can meet the resource requirements of the task request in the job configuration file, such as the number of CPU cores, the size of memory, etc. The task to be processed is scheduled to the first node, and scheduling means assigning the task to a specific node for execution on that node. The execution queue is a queue in which tasks wait to be executed. After a task is placed in the execution queue, it will wait for execution in a certain order (such as first-in first-out).

[0060] A second node with a low task priority is found in the device table based on the task priority. The task priority determines the importance and urgency of the task, and the task with a low priority can be preempted for resources. The second node is backed up with the running state of the low task priority. The backup running state is to enable the low priority task to resume execution after the resources are released. The contents of the backup may include the execution progress of the task, intermediate results, etc. The resources used by the second node are released, which means that the resources occupied by the low priority task on the node are freed up to be allocated to the pending task. The task request corresponding to the low task priority is placed in the waiting queue. The waiting queue is a queue for tasks waiting for resource allocation, and after the low priority task is preempted for resources, its task request will be placed in the waiting queue to wait for subsequent resource allocation. After the resources are released, the pending task is allocated to the second node. After the pending task is scheduled to the second node, it also needs to be placed in the execution queue to wait for execution.

[0061] The chip edge Internet of Things data scheduling method in the embodiments of the application is described above, and the chip edge Internet of Things data scheduling system in the embodiments of the application is described below. Please refer to Figure 4 An embodiment of the chip edge Internet of Things data scheduling system in the embodiments of the application includes: The receiving module 201 is configured to receive raw data from the Internet of Things device by the edge chip, pre-process the raw data, generate intermediate data, and store the intermediate data in the first cache area of the edge chip.

[0062] The association module 202 is configured to set a coordination node, initialize a device table by the coordination node, where the device table includes the device type, device state, and device running data of any edge device, collect task request information from any edge device by the coordination node, where the task request information includes the task type, working time, and task priority, and store all the task request information in a task table.

[0063] The scheduling module 203 is configured to check whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table by the coordination node, if yes, distribute the remaining tasks in the task table to the idle main edge devices, otherwise, distribute the remaining tasks to one of the idle main edge devices, active main edge devices, and idle working edge devices.

[0064] The generation module 204 is configured to extract a target node from the device table, transmit data from the edge device distributing the task to the target node, unpack and extract the intermediate data after the target node receives the data packet, pre-process the intermediate data based on the data type, and generate result data.

[0065] Through the cooperation of the above-mentioned components, first, in the task allocation process, different allocation strategies are adopted according to different conditions to realize efficient allocation and processing of tasks. By constructing a cluster to allocate tasks to the cluster, the cooperative working capacity of the cluster is fully utilized, and the efficiency of task processing is improved. In turn, the task is allocated to the idle master edge device, ensuring that the existing resources are fully utilized. By calculating the performance score of the computing device and the working time of the task, reasonable sorting and allocation are performed, and the task is allocated to the most suitable device, further improving the efficiency of task processing and the overall performance of the system. Then, by checking the working time of the active master edge device, the task is allocated to the device that can complete the current task the fastest, thereby reducing the waiting time of the task. According to the actual calculation requirements and resource consumption of the task, the complexity is evaluated, so that the device is more reasonably selected during task allocation, improving the resource utilization efficiency of the system and the task processing speed. Through the introduction of the deep learning processing model, complex task data can be better processed, and different types of Internet of Things task requirements can be adapted. Finally, in the data transmission process, different encryption methods are selected according to the data size of the data to be transmitted, and the target node performs unpacking, verifying data integrity and decryption on the received data packet, ensuring the security and reliability of the data. Based on the check code, the integrity of the data is verified to avoid incorrect or tampered data entering the subsequent processing flow, and intermediate data that can be used for subsequent processing is generated, ensuring the safe transmission and accurate processing of data in the Internet of Things system, and improving the overall security and reliability of the system.

[0066] The application can also accurately determine whether the available resources of the edge device meet the resource requirements of the task request by obtaining node resource information and job configuration files. According to different resource conditions, the method flexibly selects direct resource allocation or indirect resource allocation, ensuring that the task can be reasonably allocated resources, improving the efficiency and accuracy of task scheduling.

[0067] The application also provides a device, comprising: a memory for storing a computer program; a processor for executing the computer program to implement the above-mentioned chip edge Internet of Things data scheduling method.

[0068] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-mentioned system, system and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0069] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0070] The above-described embodiments are merely used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still make modifications to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A chip edge IoT data scheduling method, characterized in that, The chip edge Internet of Things data scheduling method comprises the following steps: The edge chip receives raw data from the Internet of Things device, pre-processes the raw data, generates intermediate data, and stores the intermediate data in a first cache area of the edge chip; A coordination node is set, which initializes a device table, wherein the device table includes the device type, device state, and device running data of any edge device, collects task request information from any edge device, and stores all the task request information in a task table, wherein the task request information includes the task type, working time, and task priority; The coordination node checks whether the number of idle master edge devices in any master edge device is greater than a first preset value based on the device table, and if so, distributes the remaining tasks in the task table to the idle master edge devices, otherwise, distributes the remaining tasks to one of the idle master edge devices, active master edge devices, and idle working edge devices; In the device table, a target node is extracted, and the edge device distributing the task transmits data to the target node, and after the target node receives the data packet, unpacks and extracts the intermediate data, pre-processes the intermediate data based on the data type, and generates result data.

2. The chip edge IoT data scheduling method of claim 1, wherein, The distribution of the remaining tasks in the task table to the idle master edge devices comprises: If there is only a task to be distributed in the task table, the idle master edge devices are constructed into a cluster, and the task to be distributed is distributed to the cluster; If the number of idle master edge devices is less than the number of remaining tasks in the task table, the remaining tasks are distributed to the idle master edge devices until the number of remaining tasks is equal to the number of idle master edge devices; If the number of idle master edge devices is greater than or equal to the number of remaining tasks, the performance score of the idle master edge devices is calculated, the remaining tasks are sorted by working time from long to short, the idle master edge devices are sorted by performance score from high to low, and the remaining tasks are distributed to the idle master edge devices.

3. The chip edge IoT data scheduling method of claim 1, wherein, The distribution of the remaining tasks to one of the idle master edge devices, active master edge devices, and idle working edge devices comprises: In the device table, it is checked whether there is an active master edge device, and if so, the task to be distributed is distributed to the active master edge device with the shortest working time, otherwise, the task to be distributed is distributed to the idle working edge device; If the number of idle working edge devices is greater than the first preset value, the first complexity of any task to be distributed is calculated, and the average complexity corresponding to all remaining tasks in the task table is calculated; If the first complexity is greater than the average complexity, the idle working edge device with the highest device performance is selected to distribute the task to be distributed, otherwise, the idle working edge device with the lowest device performance is selected to distribute the task to be distributed.

4. The chip edge IoT data scheduling method of claim 3, wherein, The calculation of the first complexity of any task to be distributed comprises: Obtaining the intermediate data corresponding to the to-be-allocated task in the first cache area based on the task type, and setting the intermediate data as task data; Performing feature extraction on the task data to generate task data features, and constructing a deep learning processing model based on the task data features; Obtaining a model architecture of the deep learning processing model, extracting model parameters based on the model architecture, and calculating the first complexity based on the model parameters.

5. The chip edge IoT data scheduling method of claim 1, wherein, The edge device that allocates the task transmits data to a target node, including: The edge device evaluates the data volume of the to-be-transmitted data, and if the data volume is greater than a second preset value, a symmetric key is generated using a random number generator, the to-be-transmitted data is encrypted using an AES algorithm to generate first encrypted data, a check code is generated for the first encrypted data, and the symmetric key, the check code, and the first encrypted data are encapsulated and transmitted to the target node; otherwise, a pair of public and private keys is generated using an RSA algorithm, the to-be-transmitted data is encrypted using the public key to generate the first encrypted data, a new check code is generated for the first encrypted data, and the first encrypted data and the new check code are encapsulated and transmitted to the target node.

6. The chip edge IoT data scheduling method of claim 5, wherein, The unpacking and extraction of the intermediate data include: The target node unpacks the received data packet, extracts encrypted data and a check code, verifies the integrity of the encrypted data based on the check code, and if the data verification fails, the target node requests the edge device to resend the data packet; otherwise, the encrypted data is decrypted based on the key in the data packet to generate the intermediate data.

7. The chip edge IoT data scheduling method of claim 1, wherein, The extraction of the target node from the device table includes: Obtaining node resource information in the device table based on the device running data, and obtaining a job configuration file of a task request in the task table based on the task priority; Judging whether the available resources of the edge devices in the device table meet the resource requirements of the task request based on the node resource information and the job configuration file, performing direct resource allocation if the available resources of at least one edge device meet the resource requirements, and performing indirect resource allocation if the available resources of any edge device do not meet the resource requirements and meet the resource requirements after occupying low-priority available resources; The edge device after resource allocation is set as the target node.

8. The chip edge IoT data scheduling method of claim 7, wherein, When performing direct resource allocation, a first node with available resources matching the job configuration file is obtained in the device table, the to-be-processed task is scheduled to the first node, and the to-be-processed task is placed in an execution queue; When performing indirect resource allocation, a second node with a low task priority is obtained in the device table based on the task priority, the second node is backed up for a low-priority running state, the use of the second node is released, a task request corresponding to the low task priority is placed in a waiting queue, the to-be-processed task is scheduled to the second node, and the to-be-processed task is placed in an execution queue.

9. A chip edge Internet of Things data scheduling system, characterized in that, The chip edge IoT data scheduling system includes: The receiving module is configured to receive raw data from the Internet of Things device by the edge chip, pre-process the raw data, generate intermediate data, and store the intermediate data in a first cache area of the edge chip; The association module is configured to set a coordination node, initialize a device table, and collect task request information from any edge device, wherein the device table includes a device type, a device state, and device running data of any edge device, the task request information includes a task type, a working time, and a task priority, and store all the task request information into a task table; The scheduling module is configured to check, by the coordination node based on the device table, whether the number of idle main edge devices in any main edge device is greater than a first preset value, if yes, distribute the remaining tasks in the task table to the idle main edge devices, otherwise, distribute the remaining tasks to one of the idle main edge devices, active main edge devices, and idle working edge devices; The generation module is configured to extract a target node from the device table, transmit data from the edge device assigned with the task to the target node, unpack and extract the intermediate data after the target node receives a data packet, pre-process the intermediate data based on a data type, and generate result data.

10. An apparatus, comprising: The memory is configured to store a computer program; The processor is configured to execute the computer program to implement the chip edge Internet of Things data scheduling method according to any one of claims 1-8. ​

Images