Chip edge internet-of-things data scheduling method, device and system

Abstract

The application relates to the technical field of Internet of Things data processing, and discloses a chip edge Internet of Things data scheduling method, device and system. The method comprises the following steps: an edge chip receives original data from an Internet of Things device, generates intermediate data, and stores the intermediate data in a first cache area of the edge chip; a coordination node is set, the coordination node initializes a device table, and stores all task request information into a task table; the coordination node checks whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table; if yes, the remaining tasks in the task table are distributed to the idle main edge devices; otherwise, the remaining tasks are distributed to one of the idle main edge devices, active main edge devices and idle working edge devices; the edge device distributing the tasks transmits data to a target node, unpacks and extracts the intermediate data, and generates result data. The application improves the efficiency and reliability of chip edge Internet of Things data scheduling.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) data processing technology, and in particular to a chip-edge IoT data scheduling method, apparatus, and system. Background Technology

[0002] With the rapid development of IoT and edge computing technologies, the amount of data generated by IoT devices is growing exponentially, making it difficult for traditional cloud computing architectures to meet the demands for real-time performance and efficiency. Edge computing, by pushing computing power to the network edge, closer to the data source, can effectively reduce data transmission latency and improve system response speed. However, edge devices have limited processing power and resources, making the efficient scheduling and management of these resources a key issue.

[0003] A similar prior art is disclosed in Chinese patent application CN114265791A, which discloses a data scheduling method, including the following steps: acquiring data to be scheduled using a first controller; in response to the destination address of the data to be scheduled being within a virtual address provided by a second controller, sequentially acquiring data from each data page of the data to be scheduled and determining the offset in the first data block of each data page based on the PRP address of each data page; sending a data processing request to the first controller using the second controller, so that the first controller sends data of a corresponding size from the current data page to the second controller according to the size of the data to be processed corresponding to the data processing request; the second controller determining the page number and the offset within the page of the data to be stored based on the virtual address range corresponding to the destination address of the corresponding size of the data and the offset of the first data block in the current data page; and writing the corresponding size of the data to be stored to the corresponding position in the data page according to the page number and the offset within the page. Another Chinese patent application, CN116483536A, discloses a data scheduling method, a computing chip, and an electronic device. The method includes: sequentially sending data requests to device memory through a first type of thread bundle, and storing the data to be processed corresponding to the data request returned by the device memory into a target area of ​​a target memory; wherein, the target memory is the memory of the thread block to which the first type of thread bundle belongs, and the target area is the area within the target memory corresponding to the data request; for any data request, if all first type of thread bundles of the thread block have stored the data to be processed corresponding to the data request into the target area of ​​the target memory, the data to be processed is retrieved from the target area of ​​the target memory into a vector register through a second type of thread bundle for processing.

[0004] The shortcomings of existing technologies mainly lie in the complex interactions between multiple controllers and the fine-grained management of data pages, which puts significant pressure on the computing and storage resources of edge devices. Multiple threads need to be allocated on the computing chip to complete data request, storage, and processing tasks. In edge IoT scenarios, edge devices typically have limited computing resources, and this high resource consumption may lead to performance bottlenecks. In practice, it is necessary to meet the actual needs of chip-based edge IoT data scheduling in multiple aspects, from data preprocessing, task allocation, and resource management to data transmission and encryption. Summary of the Invention

[0005] This application provides a chip edge IoT data scheduling method, apparatus, and system to improve the efficiency and reliability of chip edge IoT data scheduling.

[0006] Firstly, this application provides a chip edge IoT data scheduling method, the chip edge IoT data scheduling method comprising:

[0007] The edge chip receives raw data from IoT devices, preprocesses the raw data to generate intermediate data, and stores the intermediate data in the first buffer of the edge chip.

[0008] A coordination node is set up, which initializes a device table, wherein the device table includes the device type, device status, and device operation data of any edge device. The coordination node collects task request information from any edge device, wherein the task request information includes task type, working time, and task priority, and stores all the task request information in the task table.

[0009] The coordination node checks whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table. If so, the remaining tasks in the task table are assigned to the idle main edge devices. Otherwise, the remaining tasks are assigned to one of the idle main edge devices, active main edge devices, and idle working edge devices.

[0010] The target node is extracted from the device table, and the edge device that is assigned the task transmits the data to the target node. After receiving the data packet, the target node unpacks and extracts the intermediate data, preprocesses the intermediate data based on the data type, and generates the result data.

[0011] In conjunction with the first aspect, the method of allocating the remaining tasks in the task table to the idle primary edge device includes:

[0012] If the task table contains only tasks to be assigned, then the idle primary edge devices are built into a cluster, and the tasks to be assigned are assigned to the cluster.

[0013] If the number of idle primary edge devices is less than the number of remaining tasks in the task table, the remaining tasks are assigned to the idle primary edge devices until the number of remaining tasks equals the number of idle primary edge devices.

[0014] If the number of idle main edge devices is greater than or equal to the number of remaining tasks, then the performance score of the idle main edge devices is calculated, the remaining tasks are sorted from longest to shortest working time, the idle main edge devices are sorted from highest to lowest performance score, and the remaining tasks are assigned to the idle main edge devices.

[0015] In conjunction with the first aspect, the allocation of the remaining tasks to one of the idle primary edge device, the active primary edge device, and the idle working edge device includes:

[0016] Check if the active primary edge device exists in the device table. If it does, assign the task to be assigned to the active primary edge device with the shortest working time. Otherwise, assign the task to be assigned to the idle working edge device.

[0017] If the number of idle working edge devices is greater than the first preset value, then calculate the first complexity of any task to be assigned, and calculate the average complexity of all remaining tasks in the task table.

[0018] If the first complexity is greater than the average complexity, then the idle working edge device with the highest device performance is selected to assign the task to be assigned; otherwise, the idle working edge device with the lowest device performance is selected to assign the task to be assigned.

[0019] In conjunction with the first aspect, the calculation of the first complexity of any task to be assigned includes:

[0020] Based on the task type, the intermediate data corresponding to the task to be assigned is obtained from the first cache area and set as task data;

[0021] Feature extraction is performed on the task data to generate task data features, and a deep learning processing model is constructed based on the task data features;

[0022] Obtain the model architecture of the deep learning processing model, extract model parameters based on the model architecture, and calculate the first complexity based on the model parameters.

[0023] In conjunction with the first aspect, the edge device that assigns the task transmits data to the target node, including:

[0024] The edge device evaluates the amount of data to be transmitted. If the amount of data is greater than a second preset value, it uses a random number generator to generate a symmetric key, uses the AES algorithm to encrypt the data to be transmitted to generate first encrypted data, generates a checksum for the first encrypted data, and encapsulates the symmetric key, the checksum, and the first encrypted data before transmitting it to the target node. Otherwise, it uses the RSA algorithm to generate a public key and a private key pair, uses the public key to encrypt the data to be transmitted to generate the first encrypted data, generates a new checksum for the first encrypted data, and encapsulates the first encrypted data and the new checksum before transmitting it to the target node.

[0025] In conjunction with the first aspect, the unpacking and extraction of the intermediate data includes:

[0026] The target node unpacks the received data packet, extracts the encrypted data and the verification code, and verifies the integrity of the encrypted data based on the verification code. If the data verification fails, the target node requests the edge device to resend the data packet; otherwise, it decrypts the encrypted data based on the key in the data packet to generate the intermediate data.

[0027] In conjunction with the first aspect, the step of extracting the target node from the device table includes:

[0028] Based on the device operation data, node resource information is obtained from the device table; based on the task priority, the job configuration file for the task request is obtained from the task table.

[0029] Based on the node resource information and the job configuration file, it is determined whether the available resources of the edge devices in the device table meet the resource requirements of the task request. If at least one edge device has available resources that meet the resource requirements, direct resource allocation is performed. If the available resources of any edge device do not meet the resource requirements, and the edge device meets the resource requirements after occupying available resources with lower task priority, indirect resource allocation is performed.

[0030] The edge device after resource allocation is set as the target node.

[0031] In conjunction with the first aspect, a first node with available resources matching the job configuration file is obtained from the device table, the task to be processed is scheduled to the first node, and the task to be processed is placed in the execution queue; when performing indirect resource allocation, a second node with a low task priority is obtained from the device table based on the task priority, the running state of the low task priority is backed up on the second node, the resources used by the second node are released, the task request corresponding to the low task priority is placed in the waiting queue, the task to be processed is scheduled to the second node, and the task to be processed is placed in the execution queue.

[0032] Secondly, this application provides a chip edge IoT data scheduling system, the chip edge IoT data scheduling system comprising:

[0033] The receiving module is used for the edge chip to receive raw data from IoT devices, preprocess the raw data to generate intermediate data, and store the intermediate data in the first buffer of the edge chip.

[0034] The association module is used to set up a coordination node. The coordination node initializes a device table, which includes the device type, device status, and device operation data of any edge device. The coordination node collects task request information from any of the edge devices. The task request information includes task type, working time, and task priority. All the task request information is stored in the task table.

[0035] The scheduling module is used by the coordination node to check whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table. If so, the remaining tasks in the task table are assigned to the idle main edge devices; otherwise, the remaining tasks are assigned to one of the idle main edge devices, active main edge devices, and idle working edge devices.

[0036] The generation module is used to extract target nodes from the device table, and the edge devices that are assigned tasks transmit data to the target nodes. After receiving the data packets, the target nodes unpack and extract the intermediate data, preprocess the intermediate data based on the data type, and generate result data.

[0037] Thirdly, the present invention also provides an apparatus, characterized in that it comprises:

[0038] Memory, used to store computer programs;

[0039] A processor is used to implement the chip edge IoT data scheduling method described above when executing the computer program.

[0040] The technical solution provided in this application, firstly, adopts different allocation strategies based on different conditions during the task allocation process to achieve efficient task allocation and processing. By constructing a cluster, tasks are allocated to the cluster, fully leveraging the cluster's collaborative capabilities and improving task processing efficiency. Tasks are then sequentially allocated to idle master edge devices to ensure full utilization of existing resources. By calculating device performance scores and task processing times, tasks are rationally sorted and allocated to the most suitable devices, further improving task processing efficiency and overall system performance. Then, by checking the processing time of active master edge devices, tasks are allocated to the devices capable of completing the current task fastest, thereby reducing task waiting time. Complexity assessment is performed based on the actual computational requirements and resource consumption of the tasks, allowing for more rational device selection during task allocation, improving system resource utilization efficiency and task processing speed. The introduction of a deep learning processing model enables better handling of complex task data and adaptability to different types of IoT task requirements. Finally, during data transmission, different encryption methods are selected based on the size of the data to be transmitted. The target node performs operations such as unpacking, verifying data integrity, and decrypting the received data packets, ensuring data security and reliability. Data integrity is verified based on the checksum to prevent erroneous or tampered data from entering subsequent processing flows, generating intermediate data that can be used for subsequent processing. This ensures secure data transmission and accurate processing in the IoT system, improving the overall security and reliability of the system.

[0041] This application also enables accurate determination of whether the available resources of edge devices meet the resource requirements of task requests by obtaining node resource information and job configuration files. Depending on different resource conditions, the method flexibly selects either direct or indirect resource allocation to ensure that tasks receive appropriate resource allocation, thereby improving the efficiency and accuracy of task scheduling. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram of one embodiment of the chip edge IoT data scheduling method in this application.

[0044] Figure 2 This is a schematic diagram of one embodiment of the data transmission and security processing flow in this application.

[0045] Figure 3This is a schematic diagram of one embodiment of the task priority processing flow in this application.

[0046] Figure 4 This is a schematic diagram of one embodiment of the chip edge IoT data scheduling system in this application. Detailed Implementation

[0047] This application provides a chip-edge IoT data scheduling method, apparatus, and system. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the chip edge IoT data scheduling method in this application includes:

[0049] Step S101: The edge chip receives raw data from IoT devices, preprocesses the raw data to generate intermediate data, and stores the intermediate data in the first buffer of the edge chip.

[0050] It is understood that the executing entity of this application can be a chip edge IoT data scheduling device, a terminal, or a server; no specific limitation is made here. This application's embodiments use a server as an example for illustration.

[0051] Specifically, the edge chip, as the initial processing unit for data from IoT devices, receives raw data from these devices. This raw data may contain various types of information, such as temperature, humidity, and pressure data collected by sensors, and is not limited to image or audio data. The edge chip preprocesses the raw data, transforming it into intermediate data suitable for subsequent processing. For example, for image data, preprocessing may include operations such as cropping, scaling, and normalization; for sensor data, preprocessing may include data cleaning and filtering to remove noise and invalid data. The first buffer is a storage area within the edge chip used to temporarily store the preprocessed data.

[0052] Step S102: Set up a coordination node. The coordination node initializes the device table, which includes the device type, device status, and device operation data of any edge device. The coordination node collects task request information from any edge device. The task request information includes the task type, working time, and task priority. All task request information is stored in the task table.

[0053] Specifically, a coordinating node is a device node used to coordinate data. It plays a management and scheduling role in the entire system and is responsible for initializing the device table. The device table is a table that records information about all edge devices, including device type (e.g., primary edge device, working edge device), device status (e.g., idle, busy), and device operating data (e.g., remaining time for the current task, device performance metrics). The coordinating node collects task request information from any edge device. This information reflects the tasks that the edge device needs to perform. Task type refers to the type of task, such as data processing or model training; working time indicates the time required to complete the task; and task priority indicates the importance and urgency of the task for prioritization during task allocation. The coordinating node stores all collected task request information in the task table, which records and manages all pending tasks.

[0054] Step S103: The coordination node checks whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table. If so, the remaining tasks in the task table are assigned to the idle main edge devices; otherwise, the remaining tasks are assigned to one of the idle main edge devices, active main edge devices, and idle working edge devices.

[0055] Specifically, the coordinating node checks the number of idle primary edge devices in any primary edge device based on the device table to see if it exceeds a first preset value. Primary edge devices are edge devices with high computing power and management functions, while idle primary edge devices are those currently not executing any tasks. If the number of idle primary edge devices exceeds the first preset value, it means there are enough primary edge devices to handle the remaining tasks, and the coordinating node allocates the remaining tasks in the task table to idle primary edge devices. This fully utilizes the resources of primary edge devices and improves task processing efficiency. If the number of idle primary edge devices does not exceed the first preset value, it means the resources of primary edge devices are relatively scarce. In this case, the coordinating node allocates the remaining tasks to one of the following: idle primary edge devices, active primary edge devices, or idle working edge devices. Active primary edge devices are those currently executing tasks, while idle working edge devices are those with lower computing power but can assist in completing tasks. This allocation method flexibly selects appropriate devices to execute tasks based on the actual situation of different devices and the characteristics of the tasks, improving the overall performance of the system.

[0056] Step S104: Extract the target node from the device table. The edge device that assigns the task transmits the data to the target node. After receiving the data packet, the target node unpacks it and extracts the intermediate data. Based on the data type, it preprocesses the intermediate data to generate the result data.

[0057] Specifically, target nodes are extracted from the device table; these are the selected device nodes used to receive and process data. Edge devices assigned tasks transmit data to the target nodes, which can be done via wired or wireless communication. After receiving the data packets, the target node unpacks them and extracts intermediate data. Unpacking involves parsing the received data packets according to a specific format to extract the intermediate data. Then, the target node preprocesses the intermediate data based on its data type to generate result data. For example, if the intermediate data is image data, the target node might perform further image analysis and processing to extract key features and generate result data. The result data is the final data after processing.

[0058] In one specific embodiment, allocating remaining tasks in the task table to idle primary edge devices includes:

[0059] (1) If there are only tasks to be assigned in the task table, the idle main edge devices are built into a cluster and the tasks to be assigned are assigned to the cluster.

[0060] (2) If the number of idle main edge devices is less than the number of remaining tasks in the task table, the remaining tasks will be assigned to the idle main edge devices until the number of remaining tasks equals the number of idle main edge devices.

[0061] (3) If the number of idle main edge devices is greater than or equal to the number of remaining tasks, calculate the performance score of the idle main edge devices, sort the remaining tasks from longest to shortest working time, sort the idle main edge devices from highest to lowest performance score, and assign the remaining tasks to the idle main edge devices.

[0062] Specifically, the coordinating node checks the task table. If it finds only tasks to be assigned (i.e., no other assigned or being processed tasks), it builds a cluster from the idle primary edge devices. A cluster is a collaborative working unit composed of multiple primary edge devices; clustering integrates the resources of multiple devices. Tasks to be assigned are then distributed to the built cluster. The primary edge devices in the cluster can work collaboratively to complete the assigned tasks.

[0063] The coordinating node calculates the number of idle primary edge devices and the number of remaining tasks in the task table. If the number of idle primary edge devices is less than the number of remaining tasks, it means that there are not enough idle primary edge devices to allocate all the remaining tasks. The coordinating node then assigns the remaining tasks to the idle primary edge devices, one task at a time, until the number of remaining tasks equals the number of idle primary edge devices. This ensures that each idle primary edge device can be assigned a task, making full use of existing idle resources while avoiding situations where there are too many tasks to allocate.

[0064] The coordinating node calculates the performance score of idle primary edge devices. This performance score can be calculated based on various device performance metrics, such as CPU processing power, memory capacity, and storage speed. A higher performance score indicates better overall device performance. The coordinating node sorts the remaining tasks from longest to shortest processing time (the time required to complete each task). Sort tasks by processing time to prioritize longer-running tasks. The idle primary edge devices are then sorted from highest to lowest performance score, prioritizing tasks for higher-performing devices to fully utilize their resources and improve task processing speed and efficiency. Finally, the remaining tasks are assigned to the sorted idle primary edge devices, ensuring optimal task allocation based on task processing time and device performance score.

[0065] In one specific embodiment, allocating remaining tasks to one of an idle primary edge device, an active primary edge device, and an idle working edge device includes:

[0066] (1) Check if there is an active primary edge device in the device table. If it exists, assign the task to be assigned to the active primary edge device with the shortest working time. Otherwise, assign the task to be assigned to an idle working edge device.

[0067] (2) If the number of idle working edge devices is greater than the first preset value, calculate the first complexity of any task to be assigned, and calculate the average complexity of all remaining tasks in the task table.

[0068] (3) If the first complexity is greater than the average complexity, the idle working edge device with the highest device performance is selected to assign the task to be assigned; otherwise, the idle working edge device with the lowest device performance is selected to assign the task to be assigned.

[0069] Specifically, the active primary edge device refers to the primary edge device currently executing a task. Assigning tasks to the active primary edge device with the shortest working time can minimize task waiting time and improve device utilization. If no active primary edge device exists, the coordinating node will assign tasks to idle working edge devices, which are working edge devices that currently have no tasks executing.

[0070] The first complexity is a measure of computation based on factors such as the task's computational requirements and the amount of data. The average complexity is the sum of the first complexities of the remaining tasks divided by the number of remaining tasks.

[0071] If the initial complexity is greater than the average complexity, it indicates that the task to be assigned is relatively complex and requires high computing power to process. Therefore, the idle edge device with the highest device performance should be selected for assignment. If the initial complexity is less than or equal to the average complexity, it indicates that the task to be assigned is relatively simple, and an idle edge device with lower device performance can be selected for assignment to make full use of resources and avoid wasting the capabilities of high-performance devices.

[0072] In one specific embodiment, calculating the first complexity of any task to be assigned includes:

[0073] (1) Based on the task type, obtain the intermediate data corresponding to the task to be assigned in the first cache area and set it as task data.

[0074] (2) Extract features from the task data to generate task data features, and build a deep learning processing model based on the task data features.

[0075] (3) Obtain the model architecture of the deep learning processing model, extract the model parameters based on the model architecture, and calculate the first complexity based on the model parameters.

[0076] Specifically, task data refers to the intermediate data involved in executing the assigned tasks. Different task types correspond to different task data, and different task data require different processing methods.

[0077] Feature extraction is the process of extracting key information that reflects the characteristics of task data. Its purpose is to transform raw data into a more easily processed and analyzed form, reducing the dimensionality and complexity of the data. Examples include edge features and texture features in image data, or statistical features in sensor data. Deep learning processing models are models based on deep neural networks that can further process and analyze task data. The model's construction can be tailored to the task type and data characteristics, selecting appropriate network architectures. For example, convolutional neural networks (CNNs) are used for image processing, and recurrent neural networks (RNNs) are used for time series data processing.

[0078] Model architecture refers to the structure of a deep learning model, including the number of network layers, the number of neurons in each layer, activation functions, and other information. Model parameters are the weights and biases used in the deep learning model for computation and prediction; these parameters determine the model's performance and complexity. The calculation of primary complexity can be based on factors such as the number of model parameters, the number of network layers, and the computational cost per layer. For example, primary complexity can be directly proportional to the number of model parameters; more parameters mean higher model complexity. Alternatively, the computational complexity of each layer can be considered, such as the complexity of a convolutional layer, which is related to factors like kernel size and feature map size. By calculating the complexity metrics related to model parameters and architecture, the primary complexity of the task to be assigned can be obtained.

[0079] In one specific embodiment, the edge device that assigns the task transmits data to the target node, including:

[0080] The edge device assesses the amount of data to be transmitted. If the amount of data is greater than a second preset value, it uses a random number generator to generate a symmetric key, uses the AES algorithm to encrypt the data to be transmitted to generate first encrypted data, generates a checksum for the first encrypted data, and encapsulates the symmetric key, checksum, and first encrypted data before transmitting it to the target node. Otherwise, it uses the RSA algorithm to generate a public key and a private key pair, uses the public key to encrypt the data to be transmitted to generate first encrypted data, generates a new checksum for the first encrypted data, and encapsulates the first encrypted data and the new checksum before transmitting it to the target node.

[0081] Specifically, Figure 2This is a flowchart of data transmission and security processing. Data volume refers to the size of the data to be transmitted, usually measured in bytes. If the data volume is greater than a second preset value, the edge device uses a random number generator to generate a symmetric key. A symmetric key is an encryption key used for both encryption and decryption, characterized by using the same key for both. The edge device uses the AES (Advanced Encryption Standard) algorithm to encrypt the data, generating the first encrypted data. AES is a symmetric encryption algorithm with high encryption efficiency and security, suitable for encrypting large amounts of data. A checksum is generated from the first encrypted data. A checksum is an encoding used to check the integrity and accuracy of data, detecting whether errors have occurred during transmission. The symmetric key, checksum, and first encrypted data are encapsulated and transmitted to the target node. Encapsulation refers to combining multiple data elements into a single data packet for transmission. If the data volume is less than or equal to the second preset value, the edge device uses the RSA algorithm to generate a public and private key pair. RSA is an asymmetric encryption algorithm characterized by using different keys for encryption and decryption; the public key is used for encryption, and the private key is used for decryption. The edge device uses a public key to encrypt the data to be transmitted, generating the first encrypted data, and then generates a new checksum based on the first encrypted data.

[0082] In one specific embodiment, unpacking and extracting intermediate data includes:

[0083] The target node unpacks the received data packet, extracts the encrypted data and checksum, and verifies the integrity of the encrypted data based on the checksum. If the data verification fails, the target node requests the edge device to resend the data packet; otherwise, it decrypts the encrypted data based on the key in the data packet to generate intermediate data.

[0084] Specifically, unpacking refers to disassembling a packaged data packet and extracting its individual data elements, such as encrypted data, checksums, and keys. The target node extracts the encrypted data and checksum from the unpacked data. The checksum is crucial for verifying the integrity and tamper-proof nature of the data during transmission. Verification is typically performed using a verification algorithm (such as CRC checksum or hash checksum). Specifically, the target node recalculates the checksum using the same algorithm and compares it with the checksum extracted from the data packet. If the comparison fails, indicating a data verification failure, it means the encrypted data may have been corrupted or tampered with during transmission. In this case, the target node requests the edge device to resend the data packet to ensure the received data is complete and accurate. If the data verification succeeds, it means the encrypted data is complete and tamper-proof. The target node then extracts the key from the data packet. The key may be a symmetric key (such as the key used in the AES algorithm) or a private key used in asymmetric encryption (such as the private key used in the RSA algorithm). The decryption process is the reverse of the encryption process. By decrypting, the original intermediate data can be restored, that is, the data that has been preprocessed by the edge device.

[0085] In one specific embodiment, extracting the target node from the device table includes:

[0086] (1) Obtain node resource information from the device table based on device operation data, and obtain the job configuration file of the task request from the task table based on task priority.

[0087] (2) Based on node resource information and job configuration file, determine whether the available resources of the edge devices in the device table meet the resource requirements of the task request. If at least one edge device has available resources that meet the resource requirements, then perform direct resource allocation. If the available resources of any edge device do not meet the resource requirements, and the resource requirements are met after occupying the available resources of the lower task priority, then perform indirect resource allocation.

[0088] (3) Set the edge device after resource allocation as the target node.

[0089] Specifically, Figure 3 This is a flowchart for task prioritization. Device operation data reflects the current operating status of edge devices, including device load and available resources. Node resource information is recorded in the device table regarding the resource status of each edge device, such as CPU utilization, remaining memory, and storage space. The job configuration file for the task request is retrieved from the task table based on task priority. Task priority determines the urgency and importance of the task; higher-priority tasks receive priority resource allocation. The job configuration file contains detailed information such as the type and quantity of resources required by the task, such as the required number of CPU cores and memory size.

[0090] The process involves comparing the resources required in the job configuration file with the actual available resources in the node resource information. If at least one edge device has available resources that meet the task's resource requirements, direct resource allocation is performed. Direct resource allocation simply assigns the task to the edge device that meets the resource requirements; this method is simple and straightforward and suitable for situations with ample resources. If none of the edge devices has available resources that meet the task's resource requirements, but these requirements can be met by preempting available resources from lower-priority tasks, indirect resource allocation is performed. Indirect resource allocation involves adjusting the existing resource allocation, such as pausing or terminating lower-priority tasks, releasing resources, and then reallocating them to the current task. This method is suitable for situations with limited resources.

[0091] The edge devices after resource allocation are designated as target nodes. The target node is the device that ultimately receives and processes tasks; it is determined after resource allocation and is responsible for executing the tasks assigned to it.

[0092] In one specific embodiment, when performing direct resource allocation, a first node with available resources matching the job configuration file is obtained from the device table, the task to be processed is scheduled to the first node, and the task to be processed is placed in the execution queue; when performing indirect resource allocation, a second node with a lower task priority is obtained from the device table based on the task priority, the running state of the lower task priority is backed up on the second node, the resources used by the second node are released, the task request corresponding to the lower task priority is placed in the waiting queue, the task to be processed is scheduled to the second node, and the task to be processed is placed in the execution queue.

[0093] Specifically, the first node in the device table that matches the job configuration file is searched for. "Matching" here means that the available resources of the first node can meet the resource requirements requested by the task in the job configuration file, such as the number of CPU cores and memory size. The task to be processed is then scheduled to the first node. Scheduling means assigning a task to a specific node so that it can be executed on that node. The execution queue is a queue of tasks waiting to be executed. After a task is placed in the execution queue, it will wait to be executed in a certain order (e.g., first-in, first-out).

[0094] The system searches the device table for a second node with a lower task priority based on task priority. Task priority determines the importance and urgency of a task; lower-priority tasks can be preempted for resources. The running state of the lower-priority task is backed up on the second node. This backup ensures the lower-priority task can resume execution after resources are released. The backup may include task execution progress and intermediate results. The resources used by the second node are released, freeing up resources currently occupied by the lower-priority task for allocation to pending tasks. Task requests corresponding to the lower-priority task are placed in a waiting queue. This queue holds tasks awaiting resource allocation; after a lower-priority task is preempted, its request is placed in the waiting queue to await subsequent resource allocation. After resource release, pending tasks are assigned to the second node. Once scheduled to the second node, pending tasks are also placed in an execution queue to await execution.

[0095] The chip edge IoT data scheduling method in the embodiments of this application has been described above. The chip edge IoT data scheduling system in the embodiments of this application is described below. Please refer to [link / reference]. Figure 4 One embodiment of the chip edge IoT data scheduling system in this application includes:

[0096] The receiving module 201 is used for the edge chip to receive raw data from IoT devices, preprocess the raw data to generate intermediate data, and store the intermediate data in the first buffer of the edge chip.

[0097] The association module 202 is used to set up the coordination node. The coordination node initializes the device table, which includes the device type, device status, and device operation data of any edge device. The coordination node collects task request information from any edge device. The task request information includes the task type, working time, and task priority. All task request information is stored in the task table.

[0098] The scheduling module 203 is used to coordinate the node to check whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table. If so, the remaining tasks in the task table are assigned to the idle main edge devices; otherwise, the remaining tasks are assigned to one of the idle main edge devices, active main edge devices, and idle working edge devices.

[0099] The generation module 204 is used to extract the target node from the device table. The edge device that assigns the task transmits the data to the target node. After receiving the data packet, the target node unpacks it and extracts the intermediate data. Based on the data type, the intermediate data is preprocessed to generate the result data.

[0100] Through the collaborative efforts of the aforementioned components, firstly, during task allocation, different allocation strategies are adopted based on varying conditions to achieve efficient task allocation and processing. Tasks are allocated to clusters by constructing clusters, fully leveraging their collaborative capabilities to improve task processing efficiency. Tasks are then sequentially allocated to idle master edge devices to ensure full utilization of existing resources. By calculating device performance scores and task processing times, tasks are rationally sorted and allocated to the most suitable devices, further improving task processing efficiency and overall system performance. Next, by checking the processing time of active master edge devices, tasks are assigned to devices capable of completing the current task fastest, thereby reducing task waiting time. Complexity assessment is performed based on the actual computational requirements and resource consumption of the tasks, leading to more rational device selection during task allocation and improving system resource utilization efficiency and task processing speed. The introduction of deep learning processing models enables better handling of complex task data and adaptability to different types of IoT task requirements. Finally, during data transmission, different encryption methods are selected based on the size of the data to be transmitted. The target node performs operations such as unpacking, verifying data integrity, and decrypting the received data packets, ensuring data security and reliability. Data integrity is verified based on the checksum to prevent erroneous or tampered data from entering subsequent processing flows, generating intermediate data that can be used for subsequent processing. This ensures secure data transmission and accurate processing in the IoT system, improving the overall security and reliability of the system.

[0101] This application also enables accurate determination of whether the available resources of edge devices meet the resource requirements of task requests by obtaining node resource information and job configuration files. Depending on different resource conditions, the method flexibly selects either direct or indirect resource allocation to ensure that tasks receive appropriate resource allocation, thereby improving the efficiency and accuracy of task scheduling.

[0102] The present invention also provides an apparatus comprising:

[0103] Memory, used to store computer programs;

[0104] The processor is used to implement the aforementioned chip-edge IoT data scheduling method when executing computer programs.

[0105] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A chip edge IoT data scheduling method, characterized in that, The chip edge Internet of Things data scheduling method comprises the following steps: The edge chip receives raw data from the Internet of Things device, pre-processes the raw data, generates intermediate data, and stores the intermediate data in a first cache area of the edge chip; A coordination node is set, which initializes a device table, wherein the device table includes the device type, device state, and device running data of any edge device, collects task request information from any edge device, and stores all the task request information in a task table; The coordination node checks whether the number of idle main edge devices in any main edge device is greater than a first preset value based on the device table, and if so, assigns the remaining tasks in the task table to the idle main edge devices, otherwise, assigns the remaining tasks to one of the idle main edge devices, active main edge devices, and idle working edge devices; In the device table, a target node is extracted, and the edge device assigned with the task transmits data to the target node. After receiving the data packet, the target node unpacks and extracts the intermediate data, pre-processes the intermediate data based on the data type, and generates result data; Wherein, the target node is extracted in the device table, comprising: Based on the device running data, node resource information is obtained in the device table, and based on the task priority, a job configuration file of task request is obtained in the task table, which contains the required resource type and quantity of the task; Based on the node resource information and the job configuration file, it is judged whether the available resources of the edge devices in the device table meet the resource requirements of the task request. If the available resources of at least one edge device meet the resource requirements, direct resource allocation is performed. If the available resources of any edge device do not meet the resource requirements, and the available resources after occupying a low task priority meet the resource requirements, indirect resource allocation is performed, which means that the existing resource allocation is adjusted to release resources and then allocate them to the current task; The edge device after resource allocation is set as the target node.

2. The chip edge IoT data scheduling method of claim 1, wherein, The remaining tasks in the task table are assigned to the idle main edge devices, comprising: If there is only a task to be allocated in the task table, the idle main edge devices are constructed into a cluster, and the task to be allocated is assigned to the cluster; If the number of idle main edge devices is less than the number of remaining tasks in the task table, the remaining tasks are assigned to the idle main edge devices until the number of remaining tasks is equal to the number of idle main edge devices; If the number of idle main edge devices is greater than or equal to the number of remaining tasks, the performance score of the idle main edge devices is calculated, the remaining tasks are sorted by working time from long to short, the idle main edge devices are sorted by performance score from high to low, and the remaining tasks are assigned to the idle main edge devices.

3. The chip edge IoT data scheduling method of claim 1, wherein, The assigning the remaining tasks to one of the idle master edge device, the active master edge device, and the idle working edge device comprises: checking whether the active master edge device exists in the device table, if the active master edge device exists, assigning a to-be-assigned task to the active master edge device with the shortest working time, otherwise, assigning the to-be-assigned task to the idle working edge device; if the number of the idle working edge devices is greater than the first preset value, calculating a first complexity of any to-be-assigned task, and calculating an average complexity corresponding to all remaining tasks in the task table; if the first complexity is greater than the average complexity, selecting the idle working edge device with the highest device performance to assign the to-be-assigned task, otherwise, selecting the idle working edge device with the lowest device performance to assign the to-be-assigned task.

4. The chip edge IoT data scheduling method of claim 3, wherein, The calculating the first complexity of any to-be-assigned task comprises: acquiring the intermediate data corresponding to the to-be-assigned task in the first cache area based on the task type, and setting the intermediate data as task data; performing feature extraction on the task data to generate task data features, and constructing a deep learning processing model based on the task data features; acquiring a model architecture of the deep learning processing model, extracting model parameters based on the model architecture, and calculating the first complexity based on the model parameters.

5. The chip edge IoT data scheduling method of claim 1, wherein, The edge device assigning the task transmits data to a target node, comprising: The edge device evaluates the data volume of to-be-transmitted data, if the data volume is greater than a second preset value, a symmetric key is generated using a random number generator, the to-be-transmitted data is encrypted using an AES algorithm to generate first encrypted data, a check code is generated for the first encrypted data, and the symmetric key, the check code, and the first encrypted data are encapsulated and transmitted to the target node, otherwise, a pair of public key and private key is generated using an RSA algorithm, the to-be-transmitted data is encrypted using the public key to generate the first encrypted data, a new check code is generated for the first encrypted data, and the first encrypted data and the new check code are encapsulated and transmitted to the target node.

6. The chip edge IoT data scheduling method of claim 5, wherein, The unpacking and extracting the intermediate data comprises: The target node unpacks the received data packet, extracts encrypted data and a check code, verifies the integrity of the encrypted data based on the check code, if the data verification fails, the target node requests the edge device to resend the data packet, otherwise, the encrypted data is decrypted based on the key in the data packet to generate the intermediate data.

7. The chip edge IoT data scheduling method of claim 1, wherein, When the direct resource allocation is performed, a first node with available resources matched with the job configuration file is acquired in the device table, a to-be-processed task is scheduled to the first node, and the to-be-processed task is put into an execution queue. When performing indirect resource allocation, a second node with a low task priority is obtained from the device table based on the task priority, a running state of the second node is backed up with the low task priority, a used resource of the second node is released, a task request corresponding to the low task priority is put into a waiting queue, the to-be-processed task is scheduled to the second node, and the to-be-processed task is put into an execution queue.

8. A chip edge Internet of Things data scheduling system, characterized in that, The chip edge Internet of Things data scheduling system comprises: A receiving module is configured to receive raw data from an Internet of Things device by an edge chip, pre-process the raw data, generate intermediate data, and store the intermediate data in a first cache area of the edge chip. An association module is configured to set a coordination node, initialize a device table, and store all task request information in a task table, wherein the device table comprises a device type, a device state, and device running data of any edge device, the task request information comprises a task type, a working time, and a task priority, and the coordination node collects task request information from any edge device. A scheduling module is configured to check whether a number of idle main edge devices in any main edge device is greater than a first preset value based on the device table by the coordination node, and if yes, distribute remaining tasks in the task table to the idle main edge devices, and if not, distribute the remaining tasks to one of the idle main edge devices, active main edge devices, and idle working edge devices. A generation module is configured to extract a target node from the device table, transmit data to the target node by an edge device assigned with a task, unpack and extract the intermediate data after the target node receives a data packet, pre-process the intermediate data based on a data type, and generate result data.

9. An apparatus, comprising: The memory is configured to store a computer program. The processor is configured to execute the computer program to implement the chip edge Internet of Things data scheduling method according to any one of claims 1-7. The memory is configured to store a computer program. The processor is configured to execute the computer program to implement the chip edge Internet of Things data scheduling method according to any one of claims 1-7.

Images