Edge gateway computing power dynamic allocation system based on deep reinforcement learning

Abstract

The application relates to the technical field of computing power dynamic allocation, and discloses an edge gateway computing power dynamic allocation system based on deep reinforcement learning, which comprises the following steps: a southbound interface or a stacked input and output module is mapped to an entry load domain and is identified. Before protocol analysis, the system extracts the received data volume, the to-be-analyzed queue length and the earliest task deadline of each load domain, and generates an entry congestion index by fusion. Then, the index is converted into an initial computing power quota by a strategy network trained offline, and the computing power quota is compiled into a system computing power token after deducting the basic reserved computing power. A scheduling program controls protocol analysis and reasoning preprocessing tasks according to the token, and tasks exceeding the carrying limit are kept in a receiving buffer. The application makes the computing power scheduling adapt to the hardware bus beat, and improves the overall computing power response efficiency of the edge device.

Description

Technical Field

[0001] This invention relates to the field of dynamic computing power allocation technology, and more specifically, to an edge gateway dynamic computing power allocation system based on deep reinforcement learning. Background Technology

[0002] Industrial edge gateways are typically deployed in electrical cabinets, low-voltage boxes, or DIN rail locations on production sites, undertaking tasks such as southbound multi-protocol data acquisition, data normalization, edge inference, and northbound MQTT or OPCUA protocol deployment. In existing deployments, different protocol adapters, southbound drivers, caching components, and application containers within the gateway share the same set of CPU cores, memory buses, and accelerator resources. The arrival of industrial field data is often strongly constrained by the cycle time of mechanical movements or the polling cycle of the master station protocol. This leads to a concentrated surge in data traffic within a very short period of time for devices connected to the same communication bus or the same stack of input / output modules.

[0003] Existing edge computing scheduling systems mostly rely on global CPU utilization, overall queue length, or container preset priorities for dynamic resource allocation. However, this abstract allocation method from a global perspective ignores the entry phase bias phenomenon at the underlying hardware interface level. When a specific bus generates instantaneous high-concurrency traffic, the protocol parsing and data standardization tasks that enter the gateway early will preempt underlying computing resources. If the scheduling algorithm treats these concentrated tasks as ordinary and independent computational objects, it will cause blind resource skew or misallocation, resulting in massive congestion of related tasks in specific interface domains.

[0004] Furthermore, because rail-mounted edge gateways often employ fanless passive cooling designs, the thermal inertia of their casing and the enclosed air within the cabinet can cause severe thermal lag. Sustained high-load inference or protocol switching can quickly trigger processor throttling and power limits, resulting in a significant disconnect between the nominal idle CPU computing power reported by the operating system and the actual available computing power that the system can sustainably output. Under these conditions, conventional scheduling strategies based on global metrics are highly susceptible to misjudging gateways already in the thermal degradation phase as still possessing high-availability computing power, thus allowing high-load tasks to continue queuing, causing cache buildup within the gateway, and consequently crowding out the processing cycles of local high-priority alarms and real-time control commands, significantly increasing end-to-end communication and computation latency. Summary of the Invention

[0005] This invention provides a dynamic computing power allocation system for edge gateways based on deep reinforcement learning, which solves the technical problems mentioned in the background art.

[0006] This invention provides a dynamic computing power allocation system for edge gateways based on deep reinforcement learning, applied to edge gateways containing southbound physical ports, stacked input / output modules, and shared computing resources, configured to execute: The southbound physical port or the stacked input / output module is fixed as the physical inlet load domain, and the load domain identifier of the corresponding data is recorded; Before protocol parsing, extract the amount of received data, the length of the queue to be parsed, and the earliest task deadline corresponding to the physical entry load domain within the current decision window; Data arriving within the same physical ingress load domain and the same decision window are encapsulated into ingress aggregation task slices; The ingress congestion index is calculated based on the amount of data received, the length of the queue to be parsed, and the earliest task deadline, and then converted into the initial computing power quota for each physical ingress load domain through the policy network. The initial computing power quota, after deducting the gateway's basic reserved computing power, is compiled into a system computing power token. The entry aggregation task slice is executed according to the system computing power token, and tasks exceeding the token's carrying capacity are kept in its receiving buffer; At the end of the decision window, the final effective computing power quota value of each physical entry load domain is output.

[0007] The beneficial effects of this invention are as follows: By solidifying the southbound interface and stacked input / output modules into independent ingress load domains, and extracting the received data volume, the length of the queue to be parsed, and the earliest task deadline before protocol parsing to calculate the ingress congestion index, this invention combines the policy network to output the initial computing power quota of each ingress load domain and compiles it into a system computing power token. This avoids the problem of computing power allocation lag caused by concurrent access of multiple protocols, and makes the computing power scheduling mechanism highly matched with the actual communication rhythm of the underlying hardware interface of the edge gateway. Without changing the field communication protocol and peripheral automation equipment, this invention effectively alleviates the congestion and queuing phenomenon inside the gateway caused by instantaneous burst traffic, and significantly improves the overall throughput of the system and the accuracy of edge computing power allocation under high load conditions. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the overall mechanism of the edge gateway computing power dynamic allocation system based on deep reinforcement learning according to the present invention. Detailed Implementation

[0009] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0010] A deep reinforcement learning-based dynamic computing power allocation system for edge gateways is applied to edge gateways that include southbound physical ports, stacked input / output modules, and shared computing resources. The system is configured to execute: The southbound physical port or the stacked input / output module is fixed as the physical inlet load domain, and the load domain identifier of the corresponding data is recorded; Before protocol parsing, extract the amount of received data, the length of the queue to be parsed, and the earliest task deadline corresponding to the physical entry load domain within the current decision window; Data arriving within the same physical ingress load domain and the same decision window are encapsulated into ingress aggregation task slices; The ingress congestion index is calculated based on the amount of data received, the length of the queue to be parsed, and the earliest task deadline, and then converted into the initial computing power quota for each physical ingress load domain through the policy network. The initial computing power quota, after deducting the gateway's basic reserved computing power, is compiled into a system computing power token. The entry aggregation task slice is executed according to the system computing power token, and tasks exceeding the token's carrying capacity are kept in its receiving buffer; At the end of the decision window, the final effective computing power quota value of each physical entry load domain is output.

[0011] S201 reads the port enumeration results and input / output module stack information when the edge gateway starts up.

[0012] Specifically, during the edge gateway's power-on initialization phase, the operating system kernel and the underlying hardware abstraction layer scan all hardware communication buses to determine the status of currently available southbound physical ports. In some optional implementations, the hardware communication buses encompass peripheral component interconnect buses, universal serial buses, and Ethernet controller buses, with southbound physical ports including serial communication interfaces and industrial Ethernet interfaces. Further, for stacked input / output modules using backplane cascading or side-mounted expansion methods, the system polls the node addresses on the backplane communication bus to obtain the physical slot locations and activation status of each module. This physical slot location and activation status data constitutes the input / output module stack information. Finally, the port enumeration results include port type, baud rate, interrupt number, and driver mount status characteristics.

[0013] S202, generate a unique load domain number for the southbound physical port or the stacked input / output module with physical connection, forming the physical inlet load domain.

[0014] Specifically, the system identifies which southbound physical ports are connected to actual external industrial equipment based on detected link-layer activation signals or carrier sense status, and confirms which stacked I / O modules are connected to actual acquisition loops based on polling response status. In some optional implementations, for these hardware entry points with actual physical connections, the system uses an integer allocation mechanism starting from the natural number 1 and increasing in order of traversing the device tree nodes. Further, each active southbound physical port and stacked I / O module is assigned a unique positive integer, which serves as the load domain number. Finally, each load domain number and its corresponding hardware entry point constitute an independent physical entry load domain, marking the spatial boundary of the hardware access location within the software resource pool.

[0015] S203, write the load domain number into the metadata field of the protocol adapter.

[0016] Specifically, the protocol adapter is a software component responsible for converting various external industrial communication data into an internal standard format. In some optional implementations, when the runtime environment starts each protocol adapter process or thread, the configuration management component injects the corresponding assigned payload domain number as an initialization startup parameter. Further, after receiving this initialization startup parameter, the protocol adapter allocates a dedicated control block structure in its memory space and writes the payload domain number into the metadata field stored in this control block structure. Finally, when the protocol adapter receives and packages externally input data streams, it extracts the payload domain number from this metadata field as the source tag for the data packets.

[0017] S204, the load domain number is passed between the driver thread, protocol parsing thread, cache queue and edge processing container so that data entering from the same southbound physical port or stacked input / output module carries the corresponding load domain identifier.

[0018] Specifically, when external field data arrives, the corresponding driver thread at the lower level intercepts the electrical signal and converts it into a raw data frame. At this time, the driver thread appends its bound payload field number to the header of the raw data frame. In some optional implementations, the converted data stream flows to the upper layer through memory copy or shared memory mechanism and enters the protocol parsing thread. The protocol parsing thread unpacks and verifies the raw data frame according to the industrial protocol communication rules. While generating structured data, it extracts the payload field number from the header and transfers it to the data header of the structured data. Further, the structured data is then pushed into a ring-shaped buffer queue to queue for consumption. When the edge processing container running in an independent process space pulls the structured data from the buffer queue for processing, it synchronously reads the attached data header information. Finally, through the step-by-step pass-through mechanism from the driver layer to the business layer, the business data stream originating from a specific southbound physical port or a specific stacked input / output module is accompanied by the payload field identifier corresponding to its physical entry point during its system flow lifecycle.

[0019] S301, when the protocol adapter receives a data event, it reads the load field number carried by the event.

[0020] Specifically, during operation, the protocol adapter senses underlying communication activity through an asynchronous event listening mechanism or a synchronous blocking callback mechanism. In some optional implementations, when an external device sends data to the edge gateway via a communication link, causing the underlying network protocol stack or serial communication driver to trigger a data read notification, the system defines this as a protocol adapter receiving data event. Further, the protocol adapter responds to this protocol adapter receiving data event and extracts a data packet containing the complete protocol frame from the receive buffer. It then accesses a specific field in the header of this data packet, which has been permanently written with corresponding identification information during network flow. Finally, the protocol adapter directly obtains the value contained in this specific field, which is the payload field number.

[0021] S302, the received data amount is obtained by accumulating the data byte amount in the entry record area corresponding to the load field number.

[0022] Specifically, the entry record area is a dedicated storage block in memory allocated by the operating system for each active payload domain number. This storage block contains an integer accumulator variable with atomic operation characteristics. In some optional implementations, after reading the payload domain number corresponding to the current data packet, the system uses that payload domain number as an index to look up the corresponding entry record area. Further, the system calculates the number of data bytes occupied by the currently received raw data packet and uses this number as an increment value. By calling the atomic addition instruction of the underlying processor, this increment value is directly added to the integer accumulator variable inside the entry record area. Finally, during the current decision window, as the protocol adapter continues to trigger data reception events, the integer accumulator variable continuously accumulates and updates to a total value, which represents the amount of received data.

[0023] S303, read the amount of data to be processed in the parsing queue corresponding to the physical entry load domain to obtain the length of the parsing queue.

[0024] Specifically, the parsing pre-queue is a first-in, first-out circular buffer configured at the back end of the protocol adapter, used to cache raw packets that have been received but have not yet undergone network protocol unpacking and semantic recognition. In some optional implementations, each physical ingress payload domain has its own dedicated parsing pre-queue. Further, the system directly obtains the amount of data currently backed up in the parsing pre-queue by calling the queue management interface and reading the relative distance between the write and read pointers of the parsing pre-queue corresponding to that physical ingress payload domain. Finally, this amount of data reflects the total number of elements currently backed up in the buffer waiting to be processed by the protocol parsing thread; this value is the length of the parsing queue.

[0025] S304, the earliest due date is obtained by extracting the earliest due time from the set of service tasks generated by the physical ingress load domain.

[0026] Specifically, the business task set refers to a collection of computational tasks generated from data streams entering from a specific physical ingress load domain, encompassing various levels of computation, including data standardization, edge inference preprocessing, and alarm rule determination. These computational tasks are assigned a unique number corresponding to the physical ingress load domain upon creation. In some optional implementations, each computational task's control block contains a target timestamp recording its required completion time. This target timestamp is obtained by adding the system's monotonic clock time at the time of task generation to the maximum allowed latency threshold for that business type. Further, the system iterates through all task control blocks in the current business task set generated by the physical ingress load domain that are in a ready or waiting state, comparing the target timestamps in each task control block. Finally, the system retrieves the timestamp with the smallest value, which is the earliest expiration time. The absolute or relative time value represented by this earliest expiration time is the earliest task deadline.

[0027] S305, the entry record area is cleared and reconstructed at the granularity of the decision window.

[0028] Specifically, the decision window is a periodic time unit for the edge gateway to dynamically allocate computing power. Its time span is set between 2 milliseconds and 100 milliseconds, preferably 10 milliseconds. This range is determined based on the minimum polling cycle of the industrial communication bus and the task scheduling clock tick of the operating system. In some optional implementations, when the edge gateway's hardware timer generates a periodic interrupt, marking the end of the current decision window and the start of the next decision window, the computing power allocation scheduler reads the statistical data of the current decision window. Further, the computing power allocation scheduler then sends a reset command to the memory management module, which locates the entry record area corresponding to all physical entry load domains. Finally, the integer accumulation variable storing the received data is rewritten to the value 0, completing the clearing and reconstruction of the entry record area, allowing the entry record area to begin re-statistical calculations within the new decision window.

[0029] S401, at the end of the decision window, aggregate the data in the entry record area according to the load domain number.

[0030] Specifically, when the edge gateway's hardware timer generates a periodic interrupt and indicates the termination of the current time slice, the system's status acquisition module intervenes. In some optional implementations, the status acquisition module traverses the dedicated storage blocks allocated to the southbound ingress for each active state in the edge gateway's memory, reading the statistical parameters stored therein. Further, during the traversal and reading phase, the status acquisition module uses the device identifier integer as the grouping basis, grouping and organizing the total number of original arriving bytes belonging to the same identifier integer, the number of packets accumulated in the buffer, and the minimum timestamp in the service task set. Finally, a multi-dimensional data structure corresponding one-to-one with each physical access point is formed, completing the transformation from discrete records to structured grouped records.

[0031] S402, the computational tasks generated within the same physical entry load domain, including protocol parsing, data standardization, cache writing, and inference preprocessing, are bound into a single entry aggregate task slice.

[0032] Specifically, protocol parsing refers to the computational process of unpacking, verifying, and extracting fields from received raw data frames according to specific industrial communication standards. Data standardization refers to the computational process of mapping the unpacked engineering values ​​to a unified floating-point format according to the range ratio. In some optional implementations, cache writing refers to the process of serializing and storing the standardized numerical values ​​in the local memory of the edge gateway or a time-series database. Inference preprocessing refers to the preparatory work such as normalization, filtering, or tensor reconstruction performed before the data is sent to the edge artificial intelligence computing model. Further, the system scheduler instantiates an aggregate control block structure in memory and continuously pushes the thread handles or function pointers corresponding to the four computational stages with sequential dependencies originating from the same hardware entry point into the execution linked list of the aggregate control block structure. Finally, through this software encapsulation mechanism, the computational units that were originally scattered in different operating system processes or threads are reorganized into an indivisible scheduling entity, which is the entry aggregate task slice. This mechanism constrains the above-mentioned computational tasks to apply for central processing unit resources as a whole, preventing the disorderly interleaving and queuing of computational processes triggered by different hardware entry points at the system level.

[0033] S403, write the corresponding load domain number, the amount of received data, the length of the queue to be parsed, and the earliest task deadline to the entry aggregation task slice.

[0034] Specifically, after completing the memory allocation and execution list assembly of the scheduling entity, the system resource allocation module accesses the header metadata area of ​​the entry aggregation task slice. In some optional implementations, the system resource allocation module assigns each value in the multidimensional data structure organized at the end of the time slice to the corresponding variable in the header metadata area. Further, during execution, the system resource allocation module fills the identity identifier field with a positive integer representing the unique identity of the hardware interface as the load field number, fills the traffic scale field with the total number of bytes of raw packets received in the current period as the received data volume, fills the queue depth field with the total number of packets currently backed up in the buffer before unpacking as the parsing queue length, and fills the tolerance boundary field with the timestamp value extracted from the business task control block that is closest to the current system monotonic clock time interval as the earliest task deadline. Finally, after the assignment operation, the entry aggregation task slice has state parameters representing the current hardware entry congestion status and timeliness requirements when participating in subsequent computing power resource contention and quota calculation.

[0035] S501, the received data volume is converted into an equivalent number of messages to be parsed using the protocol parsing overhead weight, and then added to the length of the queue to be parsed to obtain the initial queuing load.

[0036] Specifically, the protocol parsing overhead weight is a proportional constant used to characterize the equivalent number of unparsed packets corresponding to a unit amount of data under different communication protocols. In some optional implementations, the preferred value range of this proportional constant is between 0.1 and 10.0. This preferred range is based on the fact that for simple and unencrypted serial polling protocols, the equivalent number of packets corresponding to a unit amount of data is relatively small, and the corresponding value is close to 0.1; while for industrial Ethernet protocols with deep nesting levels and security encryption verification, the equivalent number of packets corresponding to a unit amount of data is relatively large, and the corresponding value is close to 10.0. Further, the system resource allocation module extracts the protocol parsing overhead weight stored in memory, multiplies the protocol parsing overhead weight by the received data amount, and obtains the equivalent number of unparsed packets characterizing the data processing scale. Finally, the system resource allocation module adds this equivalent number of unparsed packets to the length of the unparsed queue, and the sum is the initial queuing load.

[0037] S502, the queuing urgency is assessed using the hardware binding constraint weights, the initial queuing load, and the earliest task deadline, and the entry congestion index is generated.

[0038] Specifically, the hardware binding constraint weight is a penalty coefficient used to measure the difficulty of migrating the drive path corresponding to a specific physical access point to other access points for processing. In some optional implementations, the preferred value range of this penalty coefficient is between 1.0 and 5.0. This preferred value range is based on the fact that for ordinary and replaceable general-purpose acquisition interfaces, the value is 1.0; while for dedicated high-frequency acquisition interfaces with extremely high control real-time requirements and whose physical topology cannot be changed, the value tends to be 5.0. Furthermore, to prevent arithmetic exceptions such as divisors of 0 during division calculations, the system presets a very small positive integer with a value of 0.001, which is the same as the earliest task deadline. The system adds the earliest task deadline to this very small positive integer to obtain the tolerance denominator. Finally, the system divides the initial queuing load by the tolerance denominator to obtain the queuing urgency quotient. Then, it multiplies the hardware binding constraint weight by the queuing urgency quotient, and the resulting product is the ingress congestion index. This ingress congestion index comprehensively reflects the current hardware ingress's urgent need for computing resources in a short period of time.

[0039] S503, the congestion index of each physical ingress load domain is used to construct the central processing unit system state vector and the edge inference accelerator system state vector with the total computing power of the central processing unit and the total computing power of the edge inference accelerator, respectively.

[0040] Specifically, the total computing power of the central processing unit (CPU) refers to the absolute value of globally available CPU resources that the operating system kernel allows to allocate to all southbound access points for protocol conversion and data preprocessing within the current scheduling cycle. The total computing power of the edge inference accelerator refers to the absolute value of globally available accelerator resources that the operating system kernel allows to allocate to all southbound access points for edge inference within the current scheduling cycle. In some optional implementations, the system allocates two contiguous linear storage spaces in memory as two one-dimensional array containers. Further, the system traverses all physical entry load domains and writes their respective entry congestion indices into the two one-dimensional array containers sequentially according to the order of the physical access points. Finally, after all ingress congestion indices have been written, the system appends the total computing power of the central processing unit to the end of the first one-dimensional array container and the total computing power of the edge inference accelerator to the end of the second one-dimensional array container. After continuous storage and encapsulation, the ordered sets of values ​​contained in the two one-dimensional array containers are the central processing unit system state vector and the edge inference accelerator system state vector. The two vectors constitute the global environmental observation input for the intelligent decision-making model to perform inference on the allocation of the two types of resources.

[0041] S504, input the central processing unit system state vector into the central processing unit policy network, and output the central processing unit computing power allocation action value for each of the physical entry load domains; input the edge inference accelerator system state vector into the edge inference accelerator policy network, and output the edge inference accelerator computing power allocation action value for each of the physical entry load domains.

[0042] Specifically, both the central processing unit (CPU) policy network and the edge inference accelerator policy network are artificial neural network models with a multilayer perceptron structure, pre-trained offline using reinforcement learning algorithms. This multilayer perceptron structure includes an input layer, several hidden layers, and an output layer. In some optional implementations, the number of neurons in the input layer of both policy networks is equal to the sum of the total number of physical ingress load domains and the digit 1, used to fully receive the values ​​in the corresponding system state vector. Furthermore, the hidden layers employ a fully connected structure, and each hidden layer neuron uses a linear rectified activation function for nonlinear mapping transformation. The number of neurons in the output layer is equal to the total number of physical ingress load domains, and the output layer does not use a nonlinear activation function to output an unbounded raw prediction score. Finally, the system resource allocation module sends the two constructed system state vectors into the input layer of the corresponding policy network. After matrix multiplication and addition operations of the weights of each layer of neural network and forward propagation calculation of nonlinear mapping transformation, two sets of continuous real values ​​are finally extracted from each neuron node of the two output layers. These are the central processing unit computing power allocation action value and the edge inference accelerator computing power allocation action value. Each physical entry load domain uniquely corresponds to two computing power allocation action values.

[0043] S505, using a normalized exponential function to process the computing power allocation action values ​​of each central processing unit and each edge inference accelerator respectively, to obtain the normalized allocation ratio of the central processing unit and the normalized allocation ratio of the edge inference accelerator.

[0044] Specifically, the normalized exponential function is a logic for mapping multiple arbitrary real numbers to a probability distribution with values ​​between 0 and 1, summing to the number 1. In some optional implementations, the system reads all CPU power allocation action values ​​and edge inference accelerator power allocation action values ​​respectively. Using the natural constant as the base, each power allocation action value is used as an exponent for exponentiation to obtain the natural exponent result corresponding to each power allocation action value. Further, the system sums the natural exponent results corresponding to all CPU power allocation action values ​​to obtain the global exponent sum of the CPU, and sums the natural exponent results corresponding to all edge inference accelerator power allocation action values ​​to obtain the global exponent sum of the edge inference accelerator. Finally, the system iterates through all power allocation action values ​​again, dividing the CPU natural exponent result corresponding to each physical entry load domain by the global exponent sum of the CPU to obtain the CPU normalized allocation ratio; and dividing the edge inference accelerator natural exponent result corresponding to each physical entry load domain by the global exponent sum of the edge inference accelerator to obtain the edge inference accelerator normalized allocation ratio. Both allocation ratios are between the values ​​0 and 1, and the sum of the allocation ratios corresponding to all physical ingress load domains of the same category is always equal to the number 1.

[0045] S506, the total computing power of the central processing unit is divided according to the normalized allocation ratio of the central processing unit to obtain the initial computing power quota of the central processing unit for each physical entry load domain; the total computing power of the edge inference accelerator is divided according to the normalized allocation ratio of the edge inference accelerator to obtain the initial edge inference accelerator computing power quota of each physical entry load domain.

[0046] Specifically, the system scheduler reads the total available CPU computing power and edge inference accelerator computing power for the current period. In some optional implementations, for each physical entry load domain, the system scheduler multiplies the total CPU computing power by the CPU normalization allocation ratio corresponding to that physical entry load domain, and multiplies the total edge inference accelerator computing power by the edge inference accelerator normalization allocation ratio corresponding to that physical entry load domain. Further, through multiplication, the system proportionally trims and divides the total global computing resources of both types according to the distribution weights predicted by the corresponding policy network. Finally, the two products represent the actual absolute CPU computing power allocation and the actual absolute edge inference accelerator computing power allocation values ​​that should be allocated to the currently accumulated task group at a specific hardware access point, which are the initial CPU computing power quota and the initial edge inference accelerator computing power quota, respectively. The sum of the initial CPU computing power quotas allocated to all physical ingress load domains is equal to the total CPU computing power quota, and the sum of the initial edge inference accelerator computing power quotas allocated to all physical ingress load domains is equal to the total edge inference accelerator computing power quota. This prevents the risk of deadlock or system crash due to over-allocation of computing resources in the underlying operating system of the edge gateway.

[0047] S601, Read the initial CPU computing power quota and the initial edge inference accelerator computing power quota allocated for each of the physical entry load domains.

[0048] Specifically, the token compilation module accesses specific shared memory regions or kernel object pointers to retrieve data obtained within the current decision window, processed by the normalized exponential function and output by the two policy networks. In some optional implementations, this data is stored in physical memory as two multidimensional arrays or linked lists of structures. Each physical entry load domain's load domain number uniquely corresponds to two floating-point numbers, representing the CPU computing resource share and edge inference accelerator computing resource share obtained by that hardware access location in the current scheduling cycle, i.e., the initial CPU computing power quota and the initial edge inference accelerator computing power quota. Further, the token compilation module sequentially reads these initial computing power quota values ​​into the processor's temporary registers using a cyclic clock cycle iteration instruction. Finally, it provides the underlying input data for subsequent computing power stripping and conversion compilation steps.

[0049] S602, strip the CPU base reserved computing power corresponding to system daemons, watchdog timers and secure communication to obtain the CPU remaining quota; strip the edge inference accelerator base reserved computing power corresponding to system secure communication and hardware self-test to obtain the edge inference accelerator remaining quota.

[0050] Specifically, the reserved computing power of the central processing unit (CPU) is the dedicated CPU computing resource boundary necessary to maintain the survival of the edge gateway's own system base and the stable operation of basic communication. The reserved computing power of the edge inference accelerator is the dedicated accelerator computing resource boundary necessary to maintain the security and healthy operation of the edge gateway's hardware. In some optional implementations, the system daemon is responsible for maintaining the file system status and monitoring kernel health in the background; the watchdog timer is responsible for periodically resetting the hardware to prevent gateway crashes; the secure communication function is responsible for executing data transmission encryption / decryption algorithms and access control authentication; and the hardware self-test is responsible for performing periodic health checks and calibrations of the accelerator chip. Further, the preferred value range for the reserved computing power of the CPU is set between 5% and 15% of the total CPU computing power, preferably 10%; the preferred value range for the reserved computing power of the edge inference accelerator is set between 1% and 5% of the total edge inference accelerator computing power, preferably 2%. This range is based on the premise of reserving sufficient computing resources for the base system to handle security defense and hardware reset without excessively crowding out the real-time service scheduling resource space of the southbound physical entry domain. Finally, the resource management module of the edge gateway performs subtraction calculation logic to obtain the remaining quota of the central processor by subtracting the basic reserved computing power of the central processor from the currently read initial central processor computing power quota, and to obtain the remaining quota of the edge inference accelerator by subtracting the basic reserved computing power of the edge inference accelerator from the currently read initial edge inference accelerator computing power quota.

[0051] S603, the remaining quota of the central processing unit is written into the scheduling control interface of the corresponding protocol parsing thread, data normalization thread and cache writing thread according to the load domain number; the remaining quota of the edge inference accelerator is written into the scheduling control interface of the corresponding inference preprocessing queue according to the load domain number.

[0052] Specifically, the scheduling control interface refers to the low-level configuration interface provided by the operating system kernel to limit the resource consumption limits of specific software flows. This manifests as the CPU periodic quota configuration file interface and the multi-threaded scheduling priority configuration interface in the kernel control group mechanism. In some optional implementations, the system maintains a global mapping handle structure during runtime. Using the current load domain number as an index, it locates the system process identifiers of the protocol parsing thread, data normalization thread, and cache writing thread bound to the physical entry load domain, as well as the memory buffer descriptor of the inference preprocessing queue specifically for receiving unpacked data from the hardware entry point. Further, the system calls kernel write function instructions to convert the calculated CPU remaining quota value into the corresponding CPU resource limit integer parameter and write it into the CPU periodic quota configuration file interface corresponding to the three threads mentioned above; it also converts the calculated edge inference accelerator remaining quota value into the corresponding accelerator resource limit integer parameter and writes it into the concurrent length limit interface of the inference preprocessing queue. Finally, an independent mapping channel between the hardware physical location and the two types of software control capabilities is established at the underlying operating system kernel layer.

[0053] S604, Generate a system computing power token that is valid within the decision window and includes a CPU time slice and an edge inference queue permission, wherein the CPU time slice corresponds to the CPU's remaining quota and the edge inference queue permission corresponds to the edge inference accelerator's remaining quota, and construct a computing power token table.

[0054] Specifically, the system allocates a formatted data storage block in the cache to construct a computing power token table. This computing power token table uses a headerless, contiguous linear array storage format, with its data rows sequentially associated with the load domain number, the CPU time slice value, and the number of edge inference queue permits. In some optional implementations, the system discretizes and quantizes the remaining CPU quota calculated for each load domain number, converting it into specific hardware core occupancy time to form a CPU time slice; it also discretizes and quantizes the remaining edge inference accelerator quota calculated for each load domain number, converting it into the number of concurrent slots allowed to enter the neural network chip hardware computing channel to form an edge inference queue permit. The quantized combination of these two entities constitutes the system computing power token. Further, the system sequentially fills these quantized system computing power token values ​​into the corresponding storage units of the computing power token table. Finally, the system attaches an automatic expiration attribute to the computing power token table based on the current decision window timer. This attribute limits the entire computing power token table to be a valid credential for underlying scheduling only within the clock cycle of the current decision window. Once the current decision window expires, the computing power token table will be automatically discarded and stopped being read by the system scheduler, and the computing power token table generated in a new round of loop will perform physical overwriting.

[0055] S701, At the start of the decision window, the computing power token table containing the system computing power token is loaded.

[0056] Specifically, when the edge gateway's hardware timer triggers an interrupt pulse, marking the start of a new computing power allocation time slice, the underlying system scheduler takes over supreme control of the system. In some optional implementations, the system scheduler accesses data storage blocks built into the physical memory address space in the previous computation phase, reading a contiguous linear array carrying quota parameters for each hardware access location into the CPU's internal high-speed register file. Further, through this memory addressing and high-speed read operation, the system scheduler injects the constrained CPU time slice value and the number of edge inference queue permits into the kernel scheduling context of the current clock cycle. Finally, these values, containing quantized configurations, become the underlying basis for controlling the execution pace of subsequent multi-threaded tasks.

[0057] S702, control the protocol parsing thread of each physical entry load domain to read data from the corresponding receive buffer, the amount of data read is constrained by the central processing unit time slice in the system computing power token.

[0058] Specifically, the receive buffer is a circular queue residing in the physical memory of the edge gateway, specifically responsible for temporarily storing raw, undecoded communication packets. The system scheduler dispatches a wake-up signal to the protocol parsing thread bound to a specific physical entry load domain. In some optional implementations, after being woken up, the protocol parsing thread moves the raw byte stream from its dedicated receive buffer according to a first-in-first-out (FIFO) fetching mechanism. Further, during the fetching and subsequent communication unpacking operations, the operating system kernel's performance monitoring counter records the cumulative CPU resources consumed by the protocol parsing thread in real time. Once this cumulative consumption reaches the CPU time slice value loaded into the kernel scheduling context, the system scheduler dispatches a hard interrupt instruction to the target computing core, forcibly revoking the execution privileges of the protocol parsing thread and suspending it. Finally, this preemption mechanism, linked by a hardware clock and a kernel counter, rigidly limits the total number of bytes that the protocol parsing thread can retrieve from the receive buffer to within the allocated CPU resource limit, preventing abnormal computational overload caused by unrestrained data reading.

[0059] S703 controls the data standardization and cache write thread to consume the central processing unit time slice in the corresponding system computing power token; controls the inference preprocessing thread to first consume the central processing unit time slice in the corresponding system computing power token, and then consume the edge inference queue permission in the corresponding system computing power token, with the two types of resources being accounted for independently.

[0060] Specifically, in the execution chain constructed by system instantiation, multiple computing modules in the downstream business pipeline are also subject to the overall management and control of the system scheduler. In some optional implementations, when the protocol parsing thread relinquishes its computing core due to resource deduction, the data standardization thread responsible for numerical range mapping calculation extracts the unpacked data and performs floating-point conversion. Subsequently, the cache writing thread responsible for local serialization storage pushes the data into the memory database. Then, the inference preprocessing thread responsible for tensor shape conversion prepares the input matrix for artificial intelligence network computing. Further, the data standardization thread and the cache writing thread are both classified under the same CPU kernel control group attached to the original hardware interface in the underlying operating system; the inference preprocessing thread is classified under both the CPU kernel control group and the corresponding accelerator kernel control group. When these threads call the underlying computing unit to perform operations, the system scheduler continuously decrements the CPU time slice value and the number of edge inference queue permits in two independent accounting units. Finally, through the configuration logic of multiple threads sharing the corresponding accounting unit, the derived computing tasks belonging to a specific physical access location obtain execution power from the corresponding resource quota pool.

[0061] S704, when the CPU time slice in the corresponding system computing power token is exhausted, the protocol parsing, data normalization and cache writing tasks of the physical entry load domain are suspended; when the edge inference queue permit is exhausted, the inference preprocessing task of the physical entry load domain is suspended; both types of tasks are kept in the corresponding buffer and their occupation of the corresponding system computing power tokens of other physical entry load domains is blocked.

[0062] Specifically, when the total resources consumed by the CPU's accounting unit reach the upper limit, causing the CPU's time slice to reach zero, the system scheduler sends blocking instructions to all relevant CPU computing threads derived from that physical entry load domain to perform a freeze operation. Similarly, when the total resources consumed by the accelerator's accounting unit reach the upper limit, causing the edge inference queue's permission to reach zero, the system scheduler sends blocking instructions to all relevant accelerator computing threads derived from that physical entry load domain to perform a freeze operation. In some optional implementations, communication messages that have not yet been dequeued and semi-finished task data blocks that have not completed multi-stage pipeline transformations maintain their current memory address positions and continue to queue in the original receive buffer or intermediate state storage queue, waiting for the next hardware interrupt signal to arrive for a new round of resource allocation. Furthermore, the underlying operating system's process space isolation mechanism comes into play, preventing blocked sleeping threads from having memory addressing and privilege access permissions that cross kernel control group boundaries. Finally, this interception operation based on dual isolation of memory space and process scheduling ensures that specific physical ports experiencing sudden high-frequency communication surges can only tolerate computational delays and task backlogs within their allocated quota ranges. This cuts off the possible path for high-pressure load tasks to spread outwards and seize quotas from heterogeneous physical ports, preventing system-level risks of single-point traffic overload causing overall edge gateway response paralysis.

[0063] S801, Read the computing power token table that has been completed in the decision window.

[0064] Specifically, when the hardware timer inside the edge gateway generates an interrupt signal again, indicating that the current time slice has reached its end, the system resource statistics module intervenes. In some optional implementations, the system resource statistics module sends a memory read system call instruction to the underlying operating system kernel, directly accessing the contiguous linear array address space previously allocated in physical memory for storing quota control credentials. Further, the system resource statistics module extracts from this address space the data set that guided the underlying multi-threaded computing power deduction and suspension operations within the past time slice. Finally, this data set records the initial resource values ​​allocated to each hardware access point and a snapshot of the state retained after multi-threaded concurrent consumption; this data set constitutes the completed computing power token table.

[0065] S802, extract the CPU time slice value and edge inference queue permission value of each physical entry load domain; divide the CPU time slice value by the current decision window time length to convert it into a standard CPU computing power quota; and directly use the edge inference queue permission value as the edge inference accelerator computing power quota.

[0066] Specifically, the system resource statistics module iterates through the continuous linear array extracted into the register. In some optional implementations, for each physical access point, the system resource statistics module locates its corresponding data row and separates the data segment representing the CPU time slice value and the data segment representing the number of edge inference queue permits. Further, the system resource statistics module divides the CPU time slice value by the current decision window duration to convert it into a standard processor resource allocation percentage value, which is the CPU computing power quota. Simultaneously, the system resource statistics module directly uses the number of edge inference queue permits as the occupancy quota value for the concurrent channels of the neural network acceleration chip, which is the edge inference accelerator computing power quota. Finally, for edge gateway hardware platforms that do not have a physical neural network acceleration chip, the system resource statistics module, when performing the parsing and conversion operation, directly assigns the value of the edge inference accelerator computing power quota to 0 because it cannot read the hardware support identifier.

[0067] S803, the final effective computing power quota value is formed according to the structure of the load domain number, the central processing unit computing power quota and the accelerator computing power quota.

[0068] Specifically, the system resource management module instantiates a triplet structure for data serialization in memory. In some optional implementations, the system resource management module extracts the unique positive integer identifier of each physical access point and fills it into the first field of the triplet structure corresponding to the load domain number. Further, the system resource management module then fills the calculated processor resource allocation percentage into the second field of the triplet structure corresponding to the CPU computing power quota; and fills the quota value for the concurrent channels of the neural network acceleration chip into the third field of the triplet structure corresponding to the accelerator computing power quota. Finally, after variable assignment and memory alignment operations, this data entity, with a fixed combination of three dimensions and a clear hardware control mapping relationship, becomes the final effective computing power quota value.

[0069] S804, output the final effective computing power quota value corresponding to each of the physical entry load domains to the local configuration distribution interface.

[0070] Specifically, the system resource management module packages all the constructed triplet structures into a single data frame. In some optional implementations, the system resource management module uses inter-process communication mechanisms to call the socket interface or pipe interface of the edge gateway operating system to send the single data frame to the system configuration daemon located at the application layer. Further, the system configuration daemon exposes a dedicated software call receiving channel, which is the local configuration distribution interface. Finally, after receiving the data frame, the local configuration distribution interface directly delivers the resource allocation results to the edge gateway's local log file or the background service of the visualization dashboard, achieving a one-way delivery of information for the current resource scheduling cycle. The entire data delivery action is confined to the local hardware node of the edge gateway and does not return any penalty or reward score data for updating closed-loop parameters to the neural network training module.

[0071] like Figure 1 As shown, the edge gateway in this embodiment includes multiple southbound physical inlets, which include serial ports, industrial Ethernet ports, and stacked I / O modules. The system permanently assigns each southbound physical inlet to a corresponding physical inlet load domain and sets up a receive buffer within each physical inlet load domain. Within the current decision window, before protocol parsing, the system extracts the received data volume, the length of the parsing queue, and the earliest task deadline for each physical inlet load domain, and encapsulates data arriving within the same physical inlet load domain into inlet aggregation task slices. Subsequently, the system generates an inlet congestion index based on the aforementioned state variables and inputs the inlet congestion index into the offline-trained policy network to obtain the initial computing power quota for each physical inlet load domain. The initial computing power quota, after deducting the basic reserved computing power, is compiled into a system computing power token. This system computing power token is used to constrain the execution of protocol parsing, data standardization, cache writing, and inference preprocessing. When the system computing power token for a corresponding physical inlet load domain is exhausted, unexecuted tasks remain in the receive buffer of that physical inlet load domain, preventing them from occupying system computing power tokens for other physical inlet load domains. After the decision window closes, the system outputs the final effective computing power quota value, which includes the load domain number, CPU computing power quota, and accelerator computing power quota.

[0072] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A dynamic computing power allocation system for edge gateways based on deep reinforcement learning, applied to an edge gateway comprising a southbound physical port, stacked input / output modules, and shared computing resources, characterized in that... Configured for execution: The southbound physical port or the stacked input / output module is fixed as the physical inlet load domain, and the load domain identifier of the corresponding data is recorded; Before protocol parsing, extract the amount of received data, the length of the queue to be parsed, and the earliest task deadline corresponding to the physical entry load domain within the current decision window; Data arriving within the same physical ingress load domain and the same decision window are encapsulated into ingress aggregation task slices; The ingress congestion index is calculated based on the amount of data received, the length of the queue to be parsed, and the earliest task deadline, and then converted into the initial computing power quota for each physical ingress load domain through the policy network. The initial computing power quota, after deducting the gateway's basic reserved computing power, is compiled into a system computing power token. The entry aggregation task slice is executed according to the system computing power token, and tasks exceeding the token's carrying capacity are kept in its receiving buffer; At the end of the decision window, the final effective computing power quota value of each physical entry load domain is output.

2. The edge gateway computing power dynamic allocation system based on deep reinforcement learning according to claim 1, characterized in that, The southbound physical port or the stacked input / output module is fixed as a physical inlet load domain, and the load domain identifier of the corresponding data is recorded, including: Read the port enumeration results and input / output module stack information when the edge gateway starts up; A unique load domain number is generated for the southbound physical port or the stacked input / output module with physical connection, forming the physical inlet load domain; Write the load domain number into the metadata field of the protocol adapter; The load domain number is passed between the driver thread, protocol parsing thread, cache queue, and edge processing container so that data entering from the same southbound physical port or stacked input / output module carries the corresponding load domain identifier.

3. The edge gateway computing power dynamic allocation system based on deep reinforcement learning according to claim 2, characterized in that, Before protocol parsing, extract the received data volume, parsing queue length, and earliest task deadline corresponding to the physical ingress load domain within the current decision window, including: When the protocol adapter receives a data event, it reads the payload domain number carried by the event. The received data volume is obtained by accumulating the number of data bytes in the entry record area corresponding to the load field number. The length of the parsing queue is obtained by reading the amount of data to be processed in the parsing queue corresponding to the physical entry load domain; The earliest task deadline is obtained by extracting the earliest due time from the set of service tasks generated by the physical ingress load domain. The entry record area is cleared and rebuilt at the granularity of the decision window.

4. The edge gateway computing power dynamic allocation system based on deep reinforcement learning according to claim 3, characterized in that, Encapsulating data arriving within the same physical ingress load domain and the same decision window into ingress aggregation task slices includes: At the end of the decision window, the data in the entry record area is aggregated according to the load domain number; The computational tasks generated within the same physical ingress load domain, including protocol parsing, data standardization, cache writing, and inference preprocessing, are bound into a single ingress aggregate task slice. Write the corresponding load domain number, the amount of received data, the length of the queue to be parsed, and the earliest task deadline to the entry aggregation task slice.

5. The edge gateway computing power dynamic allocation system based on deep reinforcement learning according to claim 4, characterized in that, Calculating the ingress congestion index and converting it into the initial computing power quota includes: The received data volume is calculated by using the protocol parsing overhead weight and then combined with the length of the queue to be parsed to obtain the initial queuing load. The queuing urgency is assessed using hardware binding constraint weights, the initial queuing load, and the earliest task deadline, and the ingress congestion index is generated. The system state vector is constructed by the inlet congestion index of each physical inlet load domain and the total computing power of the system; The system state vector is input into the policy network to output the computing power allocation action value for each physical entry load domain; The normalized allocation ratio is obtained by processing the computing power allocation action values ​​using the normalized exponential function; The total computing power of the system is divided according to the normalized allocation ratio to obtain the initial computing power quota for each physical entry load domain.

6. The edge gateway computing power dynamic allocation system based on deep reinforcement learning according to claim 5, characterized in that, The initial computing power quota, after deducting the gateway's basic reserved computing power, is compiled into a system computing power token, including: Read the initial computing power quota allocated to each of the physical ingress load domains; The remaining computing power is obtained by stripping the system daemon, watchdog, and secure communication gateways to retain computing power. Write the remaining quota into the scheduling control interface corresponding to the protocol parsing thread and inference preprocessing queue according to the load domain number; Generate a system computing power token that is valid within the decision window and includes a central processing unit time slice and an edge inference queue permission, and construct a computing power token table.

7. The edge gateway computing power dynamic allocation system based on deep reinforcement learning according to claim 6, characterized in that, When executing the aforementioned entry aggregation task slice, the following is included: At the start of the decision window, the computing power token table containing the system computing power token is loaded; The protocol parsing thread of each physical entry load domain is controlled to read data from the corresponding receive buffer, and the amount of data read is constrained by the system computing power token; The control data standardization, cache writing, and inference preprocessing threads sequentially consume the system computing power tokens corresponding to the same physical entry load domain. When the system computing power token is exhausted, the pending tasks corresponding to the physical entry load domain are kept in the receiving buffer, and their occupation of the system computing power tokens corresponding to other physical entry load domains is blocked.

8. The edge gateway computing power dynamic allocation system based on deep reinforcement learning according to claim 7, characterized in that, Output the final effective computing power quota value for each of the aforementioned physical ingress load domains, including: Read the computing power token table that has been completed within the decision window; Extract the CPU computing power quota and accelerator computing power quota effective for each of the physical entry load domains; The final effective computing power quota value is formed according to the structure in which the load domain number, the central processing unit computing power quota and the accelerator computing power quota are arranged accordingly. The final effective computing power quota value corresponding to each physical entry load domain is output to the local configuration distribution interface.

Images