Multi-type computing collaboration system and method based on reinforcement learning and negative feedback control

Abstract

The application discloses a multi-type computing collaborative system and method based on reinforcement learning and negative feedback control, comprising: an interface module for obtaining task information; a hierarchical collaboration module for obtaining a task processing strategy according to an initial resource context, obtaining PID hyperparameters and updating the task processing strategy according to a context analysis report; a multi-type computing module for generating task data according to the task information and sensor data, encapsulating tasks and executing task processing code according to the task processing strategy and the updated task processing strategy, processing the task data, and generating the initial resource context and the context analysis report. The application uses negative feedback hierarchical configuration adjustment to adjust the processing strategy of multi-type tasks in real time, can control the quantitative sampling parameters of data, the resource allocation parameters and the computing parameters of task processing, adapt to the dynamic changes of resource context and task context, and improves the resource utilization and the task concurrency in the system while ensuring the task processing delay and the task processing accuracy.

Description

Technical Field

[0001] This invention belongs to the field of multi-task collaborative computing technology, specifically relating to a multi-task collaborative computing system and method based on reinforcement learning and negative feedback control. Background Technology

[0002] With the widespread adoption of various sensors and the emergence of neural networks for sensor data analysis, valuable information can be extracted from data such as surveillance video streams and conference audio streams, including the opening and closing status of power grid switches, classroom head-up rates, and the number of people in a conference room. Simultaneously, with the popularization of edge computing, the high bandwidth and low computing power of edge clusters and the low bandwidth and high computing power of cloud servers in collaborative data processing systems have improved the quality of processing various types of tasks. Numerous studies have investigated computing systems that utilize neural networks to perform various analytical tasks, but these systems still face many challenges.

[0003] Currently, the mainstream multi-type collaborative computing systems include:

[0004] 1) Cooperative systems based on PID negative feedback control: These systems use the PID algorithm to calculate the deviation between the quality index of task processing and the user's expected value, and adjust the task processing parameters according to the deviation. The coordination of these systems relies on a single factor and can only adjust a single processing parameter. Moreover, in real-world scenarios, data may vary significantly, leading to large fluctuations in task processing quality. The fixed hyperparameters of the PID algorithm cannot adapt to these fluctuations, resulting in task processing quality failing to meet expectations and exhibiting poor adaptability.

[0005] 2) Reinforcement learning-based collaborative systems; these systems take system load as input and processing parameters for a single type of task as output, directly training a reinforcement learning model to adjust the processing parameters for various tasks. However, due to the large dimensionality of task processing parameters, it is difficult to train a convergent reinforcement learning model, and this model is also difficult to apply in scenarios involving other types of task processing.

[0006] Therefore, based on the above considerations, it is necessary to propose a multi-type computational collaborative system and method based on reinforcement learning and negative feedback control, so that multi-type data computation can be executed concurrently on the same platform, making full use of computer system resources and adjusting the data analysis strategies of various computational tasks in real time, thereby completing multi-type data analysis with high quality. Summary of the Invention

[0007] To address the shortcomings of the existing technologies, the present invention aims to provide a multi-type computing collaborative system and method based on reinforcement learning and negative feedback control, so as to solve the problems of poor convergence, single collaborative parameters, and poor adaptability of existing multi-type computing collaborative systems.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] The present invention provides a multi-type computing collaborative system based on reinforcement learning and negative feedback control, comprising: an interface module, a hierarchical collaborative module, and a multi-type computing module;

[0010] The interface module is used to obtain task information;

[0011] The hierarchical collaboration module is used to obtain task processing strategies based on the initial resource context sent by multiple types of computing modules; and to obtain PID hyperparameters and update task processing strategies based on the context analysis reports sent by multiple types of computing modules.

[0012] Multiple computing modules are used to generate task data based on task information and sensor data; to encapsulate tasks and execute task processing code according to the task processing strategy and updated task processing strategy sent by the hierarchical collaboration module, and to process task data; during task processing, resource context and task context are detected, and initial resource context and context analysis report are generated.

[0013] Furthermore, the interface module is used to acquire task information, which includes a data source, a task processing flow DAG = (S, E), and task processing codes and task processing delay constraints D for each stage of the task processing flow; wherein the task stage set S represents the set of task processing stages, each task processing stage s is a node in the DAG, and the task stage order set E represents the sequential relationship e = (s... i ,s j The set of ), each sequence relation e = (s i ,s j ) indicates that the task processing phase s is executed first. i The task processing code is then executed during the task processing phase. j The phased tasks.

[0014] Furthermore, the hierarchical collaboration module includes: a PID negative feedback deviation calculator, a knowledge base, a hierarchical configuration adjuster, and a reinforcement learning regulator;

[0015] The PID negative feedback deviation calculator is used to receive the task processing delay constraint D and the actual task processing delay y(t) from the scenario analysis report, and calculates the weighted deviation u(t) according to the PID formula, specifically:

[0016] u(t) = Prop + Intg + Diff

[0017] Where Prop = K p ·e(t),Intg=K i ·∫e(τ)dτ,Diff=K d·de(t) / dt,e(t)=y(t)-D,(K p ,K i ,K d ) represents the PID hyperparameter;

[0018] The knowledge base is used to store task processing strategy plans for various types of computing nodes. t ={(P s ,param s The actual processing delay y(t) after task processing, s∈S, and the task processing strategy includes the code executor P used during task phase s. s and execution parameters param s Using different computational processes and parameters to execute the task processing stage s will produce different actual task processing delays y(t), which are obtained from the scenario analysis report.

[0019] The hierarchical configuration adjuster uses the initial resource context and task information to establish an optimization model and solve for the task processing strategy plan. t Based on the knowledge base, the weighted bias u(t) is associated with the task processing strategy plan. t The parameter dimensions that need to be updated are used to obtain the update task processing strategy plan. t+1 ;

[0020] The reinforcement learning controller, based on the context analysis report, uses a reinforcement learning neural network to obtain the PID hyperparameters (K). p ,K i ,K d ).

[0021] Furthermore, the multi-type computing modules include: sensors, situation monitors, task generators, process communicators, task storage, code executors, and databases;

[0022] Sensors are used to collect sensor data, including but not limited to video data collected by cameras, audio data collected by microphones, and radio frequency data collected by antennas from radar.

[0023] The context monitor is used to detect resource contexts and obtain the initial resource context and the historical sequence of resource contexts; at the same time, it analyzes the results of task processing, detects task contexts, and obtains the historical sequence of task contexts; it merges the historical sequence of resource contexts and the historical sequence of task contexts to generate a context analysis report.

[0024] The task generator uses task information and sensor data to generate task data, and then processes the task according to the task planning strategy sent by the hierarchical collaboration module. t and update task processing strategy plan t+1Encapsulate task information and task data into a task = (DAG, s, data, plan) t A task consists of a task processing phase s, task data for phase s, a task processing flow DAG, and a task processing strategy plan. t ;

[0025] Process communicator, used in different code executors P s and task memory Q s Distribute tasks among them;

[0026] Task Memory Q s It is used to receive tasks encapsulated by the task generator and tasks distributed by the process communicator, and select the code executor according to the task processing strategy; each task processing stage s has a corresponding task memory.

[0027] code executor P s , used for the task memory Q from task processing stage s s The task is retrieved, and the code for the task processing stage is executed based on the data from that stage to process the task data and obtain the result res. s The calculation results are then encapsulated into a task task = (DAG, n, res) for the subsequent task processing stage n. s plan t The data is sent to the corresponding task memory via a process communicator; each task processing stage has a corresponding computing process.

[0028] A database is used to store the results res of task processing stage s in task processing. s .

[0029] This invention discloses a multi-type computational collaborative method based on reinforcement learning and negative feedback control. Based on the above system, the steps are as follows:

[0030] 1) Obtain task information, including data source, task processing flow DAG, task processing code at each stage of the task processing flow, and task processing delay constraint D;

[0031] 2) Based on the task storage status, code execution status, and inter-process communication status, obtain the initial resource context rt. init ;

[0032] 3) Based on the initial resource context and task information, establish an optimization model M to obtain the task processing strategy plan. t ={(P s ,param s The task processing strategy includes the code executor P used when executing the task processing code of each task processing stage s.s and execution parameters param s ;

[0033] 4) Based on the task processing strategy plan t The task processing code in task processing phase s is executed to process task data and generate the result of task processing phase {res}. s} and the tasks in the subsequent task processing phase n;

[0034] 5) During task processing, the resource context is detected, and the historical sequence of the resource context (rt) is obtained. seg Simultaneously analyze the results of all task processing stages {res s}, detect the task context and obtain the task context history sequence ctx seg The obtained resource context history sequence is merged with the task context history sequence to obtain the context analysis report rtss;

[0035] 6) Using a period of T2, a reinforcement learning neural network is used to process the context analysis report and weighted bias to obtain the PID hyperparameters (K). p ,K i ,K d );

[0036] 7) Calculate the weighted bias u(t) over period T1, and obtain the updated task processing strategy plan based on the weighted bias u(t). t+1 ;

[0037] 8) Update the task processing strategy plan t+1 Execute the task processing code for subsequent task processing stage n, and generate the result {res} for subsequent task processing stage. n} and the tasks in the subsequent task processing stage m, where n∈S;

[0038] 9) Repeat steps 5) to 8) until the task is completed.

[0039] Furthermore, step 2) specifically includes:

[0040] The initial resource context (rt) is obtained by detecting task storage, code execution, and inter-process communication. init = (qsize, v, lat, mem, bw), where the initial resource context includes, but is not limited to, the load of the task memory qsize and the task consumption v, the average computation latency of the code executor lat, the memory usage of the code executor mem, and the load of the inter-process communicator bw.

[0041] Furthermore, step 3) specifically includes:

[0042] 31) Based on the initial resource context and task information, establish a constrained optimization model, with the task processing strategy plan = (P... s ,param s ), s∈S, contains the code executor P used when executing the task processing code of each task processing stage s. s and execution parameters param s The expression for the optimized model M is as follows:

[0043]

[0044] stdelay(plan,rt init ) <D

[0045]

[0046] Where i is the index of the code executor, and S is the set of task stages; x si This indicates whether task processing phase s is executed by the i-th code executor. If task processing phase s is executed by the i-th code executor, then x si =1, otherwise x si =0; D is the task processing delay constraint; rt init Represents the initial resource context; acc(plan,rt) init ) represents the precision function of the task processing strategy, specifically as follows: ε s (param s ,rt init This represents the functional relationship between the computational parameters of task processing stage s and the initial resource context on the processing accuracy of task processing stage s. This represents the functional relationship between the computational parameters of task processing phase s and the initial resource context on the processing accuracy of task processing phase s; delay(plan,rt) init The delay function () represents the time delay of the task processing strategy, specifically: delay(plan, rt) init )=∑ s ∑ i x si ·[proc si (param s ,rt init )+transfer si (param s ,rt init )]·α s proc si (param s ,rt init) represents the functional relationship between the calculation parameter of task processing stage s and the initial resource context with respect to the processing delay of task processing stage s, transfer si (param s ,rt init ) represents the functional relationship between the calculation parameter of task processing stage s and the initial resource context with respect to the processing accuracy of task processing stage s, α s represents the scaling ratio of the calculation parameter of task processing stage s to the average processing delay;

[0047] 32) solving the optimization model M by using an enumeration method to obtain the task processing strategy plan t ; specifically: traversing each parameter combination of the task processing strategy, substituting the candidate task processing strategy plan0 and the initial resource context rt init into the delay function to obtain the delay delay(plan0,rt init ); if delay(plan0,rt init )<D is satisfied, substituting the parameter combination into the accuracy function to obtain the accuracy acc(plan0,rt init ); selecting the candidate task processing strategy with the maximum acc(plan0,rt init ) as the task processing strategy plan t .

[0048] Further, step 4) specifically comprises:

[0049] 41) according to the task processing strategy plan t wherein the calculation parameter param s , sampling and quantifying sensor data according to a specific quantization parameter r and a sampling rate fps to obtain task data data, and encapsulating the task data and task information into a task task s =(DAG,s,data,plan t ), the task comprises a task processing stage s, task data data, a task processing flow DAG and a task processing strategy plan t ;

[0050] 42) storing the encapsulated task task by using the task memory Q corresponding to the task processing stage s s s ;;

[0051] 43) according to the task processing strategy plan t , selecting a code executor, wherein the code executor loads task processing code of the task processing stage s according to task s , executes the task processing code by using the calculation parameter in the task processing strategy, and processes the task data to obtain a processing result res s ​The data is then saved to the database, and the task for the subsequent task processing stage n is encapsulated. n =(DAG,n,res) s plan t );

[0052] 44) Task n Distribute to the task memory corresponding to task processing stage n.

[0053] Furthermore, step 5) specifically includes:

[0054] 51) Periodically check resource conditions, including but not limited to the load status of the task memory (qsize), task consumption (v), average computation latency of the code executor (lat), memory usage of the code executor (mem), and load status of the inter-process communicator (bw). Retain the resource conditions for each check as rt = (qsize, v, lat, mem, bw), forming a historical sequence of resource conditions rt. seg =(rt0,rt1,rt2,...,rt n );

[0055] 52) Periodically detect task scenarios and analyze the task processing results stored in the database {res} s}, s∈S, processing result res s The sum of the winning bid data values ​​n res and the dimension d of the vector data values res The task context ctx = (n) res ,d res The task context of each analysis is preserved to form a task context history sequence ctx. seg =(ctx0,ctx1,ctx2,...,ctx) n ), ctx seg The actual processing delay y of the task is extracted and merged to form the actual processing delay sequence y(t);

[0056] 53) Merge the resource context history sequence and the task context history sequence to obtain the context analysis report rtss = (rt seg ,ctx seg ).

[0057] Furthermore, step 6) specifically includes:

[0058] The context analysis report rtss is received at a period of T2 and input into the reinforcement learning neural network f(rtss) = (K p ,K i ,K d From this, the PID hyperparameters (K) are obtained. p ,Ki ,K d ).

[0059] Further, step 7) specifically includes:

[0060] 71) Based on the task execution delay constraint D, the actual task processing delay sequence y(t) in the scenario analysis report, and the PID hyperparameter (K) with period T1, the task execution delay is determined by period T1. p ,K i ,K d The weighted deviation u(t) is calculated according to the PID formula u(t) = Prop + Intg + Diff.

[0061] 72) Utilize task processing strategies (plans) t ={(P s ,param s The actual processing latency of the task is used to obtain the trend of the relative latency of the k-th dimension of the task processing strategy parameter. The parameter dimension includes the code executor P used in the task processing stage s. s and calculation parameters;

[0062] 73) Within the range of the minimum to the maximum value of the weighted deviation u(t), divide the data into L intervals, and then analyze the changing trends in the knowledge base. After sorting, it is divided into L sets on average, with each set containing... The largest strategy parameter dimension k is the adjustment dimension of the set; the L intervals of the weighted bias are mapped to the strategy adjustment dimensions of the L sets of the knowledge base to obtain the mapping between bias and strategy;

[0063] 74) Locate the deviation interval to which the current weighted deviation u(t) belongs in the mapping between deviation and policy, thereby finding the corresponding task processing policy parameter dimension, adjusting the value of this dimension, and obtaining the updated task processing policy plan. t+1 Update the task processing strategy plan t+1 ={(P s ,param s )},s∈S, contains the code executor P used when executing the task processing code of each subsequent task processing stage n. s and execution parameters param s .

[0064] Furthermore, step 8) specifically includes:

[0065] 81) According to the updated task processing strategy plan t+1 Select a code executor, and then the code executor will handle n tasks in the subsequent task processing phase. nLoad the task processing code for subsequent task processing stage n, execute the task processing code using the calculation parameters in the updated task processing strategy, and then process the result res. n Save to the database, and simultaneously encapsulate the task of m in the subsequent task processing stage. m =(DAG,m,res) n plan t+1 );

[0066] 82) Task m Distribute to the task memory corresponding to task processing stage m.

[0067] The beneficial effects of this invention are:

[0068] 1. This invention uses a negative feedback hierarchical configuration adjustment mechanism to adjust the processing strategies of multiple types of tasks in real time. It can control parameters such as data quantization sampling parameters, task processing resource allocation parameters, and calculation parameters, adapting to dynamic changes in resource and task situations. While ensuring task processing latency and accuracy, it improves the resource utilization and task concurrency in the system.

[0069] 2. In designing the collaborative mechanism, this invention considers using a reinforcement learning neural network to regulate the PID hyperparameters and using a knowledge base to obtain the mapping between bias and policy. Based on the bias-policy mapping and the weighted bias calculated by the PID algorithm, the corresponding task processing policy parameters are adjusted. The output parameters of the reinforcement learning neural network have a very low dimensionality, making the model easy to train and converge. Furthermore, the mapping between bias and policy allows for finer-grained adjustment of the task processing policy, improving the system's resource utilization.

[0070] 3. This invention considers both resource context and task context as input to the reinforcement learning neural network, and adjusts the PID hyperparameters in a timely manner using the reinforcement learning neural network, so that the PID algorithm can adapt to the dynamic changes of the real scene and converge quickly in various scenarios. Attached Figure Description

[0071] Figure 1 This is a structural example diagram of the system of the present invention.

[0072] Figure 2 This is an example diagram of the hierarchical collaboration module in this invention.

[0073] Figure 3 This is an example diagram of the hierarchical configuration adjuster in this invention.

[0074] Figure 4 This is a flowchart of the method of the present invention. Detailed Implementation

[0075] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0076] Reference Figure 1 As shown, the present invention provides a multi-type computing collaborative system based on reinforcement learning and negative feedback control, comprising: an interface module, a hierarchical collaborative module, and a multi-type computing module;

[0077] The interface module is used to obtain task information;

[0078] The hierarchical collaboration module is used to obtain task processing strategies based on the initial resource context sent by multiple types of computing modules; and to obtain PID hyperparameters and update task processing strategies based on the context analysis reports sent by multiple types of computing modules.

[0079] Multiple computing modules are used to generate task data based on task information and sensor data; to encapsulate tasks and execute task processing code according to the task processing strategy and updated task processing strategy sent by the hierarchical collaboration module, and to process task data; during task processing, resource context and task context are detected, and initial resource context and context analysis report are generated.

[0080] The interface module is used to acquire task information, which includes a data source, a task processing flow DAG = (S, E), task processing codes for each stage of the task processing flow, and task processing delay constraints D. The task stage set S represents the set of task processing stages, with each task processing stage s serving as a node in the DAG. The task stage order set E represents the sequential relationship between each task processing stage e = (s...). i ,s j The set of ), each sequence relation e = (s i ,s j ) indicates that the task processing phase s is executed first. i The task processing code is then executed during the task processing phase. j The phased tasks.

[0081] For example, video stream analysis requires face detection, pose estimation, and head-up rate statistics, totaling three processing stages. The set S includes face detection s0, pose estimation s1, and head-up rate statistics s2, and E includes (s0, s1) and (s1, s2). (s0, s1) indicates that face detection is performed first, followed by pose estimation, and (s1, s2) indicates that pose estimation is performed first, followed by head-up rate statistics. Similarly, radio frequency (RF) data stream analysis requires Fourier transform, event segmentation, and action classification, totaling three processing stages. The set S includes Fourier transform s3, event segmentation s4, and action classification s5, and E includes (s3, s4) and (s4, s5). (s3, s4) indicates that Fourier transform is performed first, followed by event segmentation, and (s4, s5) indicates that event segmentation is performed first, followed by action classification.

[0082] Reference Figure 2 As shown, the hierarchical collaboration module includes: a PID negative feedback deviation calculator, a knowledge base, a hierarchical configuration adjuster, and a reinforcement learning regulator;

[0083] The PID negative feedback deviation calculator is used to receive the task processing delay constraint D and the actual task processing delay y(t) from the scenario analysis report, and calculates the weighted deviation u(t) according to the PID formula, specifically:

[0084] u(t) = Prop + Intg + Diff

[0085] Where Prop = K p ·e(t),Intg=K i ·∫e(τ)dτ,Diff=K d ·de(t) / dt,e(t)=y(t)-D,(K p ,K i ,K d ) represents the PID hyperparameter;

[0086] The knowledge base is used to store task processing strategy plans for various types of computing nodes. t ={(P s ,param s The actual processing delay y(t) after task processing, s∈S, and the task processing strategy includes the code executor P used during task phase s. s and execution parameters param s Using different computational processes and parameters to execute the task processing stage s will produce different actual task processing delays y(t), which are obtained from the scenario analysis report.

[0087] The hierarchical configuration adjuster uses the initial resource context and task information to establish an optimization model and solve for the task processing strategy plan. t Based on the knowledge base, the weighted bias u(t) is associated with the task processing strategy plan. t The parameter dimensions that need to be updated are used to obtain the update task processing strategy plan. t+1 ;

[0088] The reinforcement learning controller, based on the context analysis report, uses a reinforcement learning neural network to obtain the PID hyperparameters (K). p ,K i ,K d ).

[0089] The multi-type computing modules include: sensors, situation monitors, task generators, process communicators, task storage, code executors, and databases.

[0090] Sensors are used to collect sensor data, including but not limited to video data collected by cameras, audio data collected by microphones, and radio frequency data collected by antennas from radar.

[0091] The context monitor is used to detect resource contexts and obtain the initial resource context and the historical sequence of resource contexts; at the same time, it analyzes the results of task processing, detects task contexts, and obtains the historical sequence of task contexts; it merges the historical sequence of resource contexts and the historical sequence of task contexts to generate a context analysis report.

[0092] The task generator uses task information and sensor data to generate task data, and then processes the task according to the task planning strategy sent by the hierarchical collaboration module. t and update task processing strategy plan t+1 The task information and task data are encapsulated into a task task = (DAG, s, data, plan) t A task consists of a task processing phase s, task data for phase s, a task processing flow DAG, and a task processing strategy plan. t ;

[0093] Process communicator, used in different code executors P s and task memory Q s Distribute tasks among them;

[0094] Task Memory Q s It is used to receive tasks encapsulated by the task generator and tasks distributed by the process communicator, and select the code executor according to the task processing strategy; each task processing stage s has a corresponding task memory.

[0095] code executor P s, used for the task memory Q from task processing stage s s The task is retrieved, and the code for the task processing stage is executed based on the data from that stage to process the task data and obtain the result res. s The calculation results are then encapsulated into a task task = (DAG, n, res) for the subsequent task processing stage n. s plan t The data is sent to the corresponding task memory via a process communicator; each task processing stage has a corresponding computing process.

[0096] A database is used to store the results res of task processing stage s in task processing. s .

[0097] Reference Figure 4 As shown, this invention discloses a multi-type computational collaborative method based on reinforcement learning and negative feedback control. Based on the above system, the steps are as follows:

[0098] 1) Obtain task information, including data source, task processing flow DAG, task processing code at each stage of the task processing flow, and task processing delay constraint D;

[0099] 2) Based on the task storage status, code execution status, and inter-process communication status, obtain the initial resource context rt. init Specifically, it includes:

[0100] 21) Detect task storage status, code execution status, and inter-process communication status to obtain the initial resource context rt. init = (qsize, v, lat, mem, bw), where the initial resource context includes, but is not limited to, the load of the task memory qsize and the task consumption v, the average computation latency of the code executor lat, the memory usage of the code executor mem, and the load of the inter-process communicator bw (e.g., uplink and downlink bandwidth).

[0101] 3) Based on the initial resource context and task information, establish an optimization model M to obtain the task processing strategy plan. t ={(P s ,param s The task processing strategy includes the code executor P used when executing the task processing code of each task processing stage s. s and execution parameters param s Specifically, it includes:

[0102] 31) Based on the initial resource context and task information, establish a constrained optimization model, with the task processing strategy plan = (P... s ,params ), s∈S, contains the code executor P used when executing the task processing code of each task processing stage s. s and execution parameters param s The expression for the optimized model M is as follows:

[0103]

[0104] stdelay(plan,rt init ) <D

[0105]

[0106] Where i is the index of the code executor, and S is the set of task stages; x si This indicates whether task processing phase s is executed by the i-th code executor. If task processing phase s is executed by the i-th code executor, then x si =1, otherwise x si =0; D is the task processing delay constraint; rt init Represents the initial resource context; acc(plan,rt) init ) represents the precision function of the task processing strategy, specifically as follows: ε s (param s ,rt init This represents the functional relationship between the computational parameters of task processing stage s and the initial resource context on the processing accuracy of task processing stage s. This represents the functional relationship between the computational parameters of task processing phase s and the initial resource context on the processing accuracy of task processing phase s; delay(plan, rt init The delay function () represents the time delay of the task processing strategy, specifically: delay(plan, rt) init )=∑ s ∑ i x si ·[proc si (param s ,rt init )+transfer si (param s ,rt init )]·α s proc si (param s ,rt init The expression ) represents the functional relationship between the computational parameters of task processing phase s and the initial resource context on the processing latency of task processing phase s. si (param s ,rtinit ) represents the functional relationship between the calculation parameters of task processing stage s and the initial resource context on the processing accuracy of task processing stage s, α s represents the scaling ratio of the calculation parameter of task processing stage s to the average processing delay;

[0107] 32) Solving the optimization model M by enumeration method to obtain the task processing strategy plan t ; specifically: traversing each parameter combination of the task processing strategy, and combining the alternative task processing strategy plan0 and the initial resource context rt init is substituted into the delay function to obtain the delay delay(plan0,rt init ); if delay(plan0,rt init )<D, substituting the parameter combination into the accuracy function to obtain the accuracy acc(plan0,rt init ; selecting the alternative task processing strategy with the largest acc(plan0,rt init ) as the task processing strategy plan t .

[0108] For example: acc(plan,rt init ) can be expressed as:

[0109]

[0110] wherein ε s (fps,rt init ) represents the functional relationship between the data sampling rate fps and the initial resource context rt init on the processing accuracy of task processing stage s, represents the functional relationship between the data quantization parameter r and the initial resource context rt init on the processing accuracy of task processing stage s; these functional relationships can be linear or non-linear (in this case, variables can be discretized and the functional relationship can be obtained by enumeration fitting), and the main purpose is to calculate the estimated task processing delay and task accuracy of task processing stage s under the current input data sampling rate, quantization parameters and current system context through indicators such as sampling rate, resource context and quantization parameters, so as to help solve the optimization model.

[0111] For example: delay(plan,rt init ) can be expressed as:

[0112] delay(plan,rt init )=∑ s ∑ i x si ·[proc si (r,rt init)+transfer si (r,rt init )]·α fps ,

[0113] Among them, proc si (r,rt init ) represents the quantization parameter r and the initial resource context rt. init The functional relationship between the processing delay of task processing stage s and transfer. si (r,rt init ) represents the functional relationship between the quantization parameter r and the initial resource context and the processing latency of the task processing stage s; α fps This represents the scaling ratio of the average processing delay to different sampling rates; similarly, these functional relationships can be linear or nonlinear (in which case the variables can be discretized and the functional relationship can be obtained by enumeration and fitting).

[0114] 4) Based on the task processing strategy plan t The task processing code in task processing phase s is executed to process task data and generate the result of task processing phase {res}. s} and the tasks in the subsequent task processing phase n; specifically including:

[0115] 41) According to the task processing strategy plan t The calculation parameter param s Sensor data is sampled and quantized according to a specific quantization parameter r and a sampling rate fps to obtain task data, which is then encapsulated with task information to form a task. s = (DAG, s, data, plan) t The task includes the task processing phase s, task data data, task processing flow DAG, and task processing strategy plan. t ;

[0116] 42) Utilize the task memory Q corresponding to task processing stage s s Save the packaged task s ;;

[0117] 43) According to the task processing strategy plan t Select a code executor; the code executor will then select the task. s Load the task processing code in task processing phase s, execute the task processing code using the calculation parameters in the task processing strategy, and process the task data to obtain the processing result res. s The data is then saved to the database, and the task for the subsequent task processing stage n is encapsulated. n =(DAG,n,res) s plant );

[0118] 44) Task n Distribute to the task memory corresponding to task processing stage n.

[0119] 5) During task processing, the resource context is detected, and the historical sequence of the resource context (rt) is obtained. seg Simultaneously analyze the results of all task processing stages {res s}, thus obtaining the task context history sequence ctx seg The obtained resource context history sequence is merged with the task context history sequence to obtain the context analysis report RTSS; specifically, it includes:

[0120] 51) Periodically check resource conditions, including but not limited to the load status of the task memory (qsize), task consumption (v), average computation latency of the code executor (lat), memory usage of the code executor (mem), and load status of the inter-process communicator (bw). Retain the resource conditions for each check as rt = (qsize, v, lat, mem, bw), forming a historical sequence of resource conditions rt. seg =(rt0,rt1,rt2,...,rt n );

[0121] 52) Periodically detect task scenarios and analyze the task processing results stored in the database {res} s}, s∈S, processing result res s The sum of the winning bid data values ​​n res and the dimension d of the vector data values res The task context ctx = (n) res ,d res The task context of each analysis is preserved to form a task context history sequence ctx. seg =(ctx0,ctx1,ctx2,...,ctx n ), ctx seg The actual processing delay y of the task is extracted and merged to form the actual processing delay sequence y(t);

[0122] For example, in the process of radar radio frequency data analysis, it is necessary to go through the steps of Fourier transform, event segmentation, and action classification. Event segmentation is based on the feature map generated by Fourier transform, and the segmentation result includes the number of events n. event Each event contains a graph (img). i Each graph has n channels. channel Length n len Width n width and the number of pixels n pixelThen, the task context ctx obtained by the context analyzer for the event segmentation steps is (n event ,n channel ,n len ,n width ,n pixel );

[0123] 53) Merge the resource context history sequence and the task context history sequence to obtain the context analysis report rtss = (rt seg ,ctx seg ).

[0124] 6) Using a period of T2, a reinforcement learning neural network is used to process the context analysis report and weighted bias to obtain the PID hyperparameters (K). p ,K i ,K d Specifically, this includes:

[0125] The context analysis report rtss is received at a period of T2 and input into the reinforcement learning neural network f(rtss) = (K p ,K i ,K d From this, the PID hyperparameters (K) are obtained. p ,K i ,K d );

[0126] 7) Calculate the weighted bias u(t) over period T1, and obtain the updated task processing strategy plan based on the weighted bias u(t). t+1 ;reference Figure 3 As shown; specifically including:

[0127] 71) Based on the task execution delay constraint D, the actual task processing delay sequence y(t) in the scenario analysis report, and the PID hyperparameter (K) with period T1, the task execution delay is determined by period T1. p ,K i ,K d The weighted deviation u(t) is calculated according to the PID formula u(t) = Prop + Intg + Diff.

[0128] 72) Utilize task processing strategies (plans) t ={(P s ,param s The actual processing latency of the task is used to obtain the trend of the relative latency of the k-th dimension of the task processing strategy parameter. The parameter dimension includes the code executor P used in the task processing stage s. s and calculation parameters;

[0129] 73) Within the range of the minimum to the maximum value of the weighted deviation u(t), divide the data into L intervals, and then analyze the changing trends in the knowledge base. After sorting, it is divided into L sets on average, with each set containing... The largest strategy parameter dimension k is the adjustment dimension of the set; the L intervals of the weighted bias are mapped to the strategy adjustment dimensions of the L sets of the knowledge base to obtain the mapping between bias and strategy;

[0130] For example, in video stream analysis, the strategy parameters include video sampling rate, video frame rate, and the computing power of the code executor. The latency change trend can be sorted according to the three parameter dimensions, which correspond to the three intervals of the weighted deviation.

[0131] 74) Locate the deviation interval to which the current weighted deviation u(t) belongs in the mapping between deviation and policy, thereby finding the corresponding task processing policy parameter dimension, adjusting the value of this dimension, and obtaining the updated task processing policy plan. t+1 Update the task processing strategy plan t+1 ={(P s ,param s )},s∈S, contains the code executor P used when executing the task processing code of each subsequent task processing stage n. s and execution parameters param s .

[0132] 8) Update the task processing strategy plan t+1 Execute the task processing code for subsequent task processing stage n, and generate the result {res} for subsequent task processing stage. n} and the tasks in the subsequent task processing stage m, where n∈S; specifically including:

[0133] 81) According to the updated task processing strategy plan t+1 Select a code executor, and then the code executor will handle n tasks in the subsequent task processing phase. n Load the task processing code for subsequent task processing stage n, execute the task processing code using the calculation parameters in the updated task processing strategy, and then process the result res. n Save to the database, and simultaneously encapsulate the task of m in the subsequent task processing stage. m =(DAG,m,res) n plan t+1 );

[0134] 82) Task m Distribute to the task memory corresponding to task processing stage m.

[0135] 9) Repeat steps 5) to 8) until the task is completed.

[0136] This invention has many specific applications. The above description is only a preferred embodiment of this invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of this invention, and these improvements should also be considered within the scope of protection of this invention.

Claims

1. A multi-type computational collaborative system based on reinforcement learning and negative feedback control, characterized in that, include: Interface module, hierarchical collaboration module, multi-type computing module; The interface module is used to obtain task information; The hierarchical collaboration module is used to obtain task processing strategies based on the initial resource context sent by multiple types of computing modules; Based on the scenario analysis reports sent by the various computing modules, the PID hyperparameters and updated task processing strategies are obtained. Multiple types of computing modules are used to generate task data based on task information and sensor data; and to encapsulate tasks and execute task processing code based on the task processing strategy sent and updated by the hierarchical collaboration module to process task data. During task processing, resource context and task context are detected, and an initial resource context and context analysis report are generated; The hierarchical collaboration module includes: a PID negative feedback deviation calculator, a knowledge base, a hierarchical configuration adjuster, and a reinforcement learning regulator; The PID negative feedback deviation calculator is used to receive task processing delay constraints. Actual processing latency of tasks in the context analysis report Calculate the weighted deviation according to the PID formula. Specifically: ； in, , , , , These are PID hyperparameters; The knowledge base is used to store task processing strategies for various types of computing nodes. And the actual processing delay of the task after processing the task , The task processing strategy includes the task execution phase. The code executor used at that time and execution parameters Different computational processes and parameters are used to execute the task processing phase. Different actual task processing delays will occur. Actual task processing delay Obtained from the context analysis report; The hierarchical configuration adjuster uses the initial resource context and task information to establish an optimization model and solve for the task processing strategy. Based on the knowledge base, the weighted bias is... Related to task processing strategy The parameter dimensions that need to be updated are used to determine the update task processing strategy. ; The reinforcement learning controller, based on the context analysis report, uses a reinforcement learning neural network to obtain PID hyperparameters. ; Among them, the weighted bias Related to task processing strategy The parameter dimensions that need to be updated are used to determine the update task processing strategy. Specifically, it includes; In a cycle Based on task execution delay constraints Actual processing delay sequence of tasks in the scenario analysis report and PID hyperparameters According to the PID formula Calculate weighted bias ; Utilizing task processing strategies The actual processing delay of the task is used to obtain the task processing strategy parameter. Trends in relative time delay across various dimensions ; Parameter dimensions include task processing stages The code executor and computation parameters used; Weighted bias Divide within the range of minimum to maximum value. Each interval will represent the changing trends within the knowledge base. After sorting, divide equally into There are sets, each set contains Largest strategy parameter dimension Adjust the dimension of this set; weight the bias. Each interval and knowledge base The policy adjustment dimensions of each set correspond to the bias and the policy, thus obtaining the mapping between the bias and the policy; Find the current weighted bias in the mapping between bias and strategy. By identifying the corresponding deviation range, the appropriate task processing strategy parameter dimension can be determined. Adjusting the value of this dimension yields the updated task processing strategy. Update task processing strategy This includes the execution of each subsequent task processing stage. The code executor used when processing task code and execution parameters .

2. The multi-type computational collaborative system based on reinforcement learning and negative feedback control according to claim 1, characterized in that, The interface module is used to obtain task information, which includes data source and task processing flow. And the task processing code and task processing latency constraints at each stage of the task processing flow. The task phase set This represents a set of task processing stages, each task processing stage... As A node represents the set of task phase sequences. Indicates the sequential relationship between the various task processing stages. A set, each with a sequential relationship This indicates that the task processing phase will be executed first. The task processing code is executed during the task processing phase. The phased tasks.

3. The multi-type computational collaborative system based on reinforcement learning and negative feedback control according to claim 1, characterized in that, The multi-type computing modules include: sensors, situation monitors, task generators, process communicators, task storage, code executors, and databases; Sensors are used to collect sensor data, including but not limited to video data collected by cameras, audio data collected by microphones, and radio frequency data collected by antennas from radar. The context monitor is used to detect resource contexts and obtain the initial resource context and the historical sequence of resource contexts; at the same time, it analyzes the results of task processing, detects task contexts, and obtains the historical sequence of task contexts; it merges the historical sequence of resource contexts and the historical sequence of task contexts to generate a context analysis report. The task generator uses task information and sensor data to generate task data, and then processes the data according to the task handling strategy sent by the hierarchical collaboration module. and update task processing strategies Encapsulate task information and task data into a task A task includes a task processing phase. Task processing phase Task data Task processing flow and task processing strategies ; A process communicator is used to distribute tasks between different code executors and task stores; The task store receives tasks encapsulated by the task generator and tasks distributed by the process communicator, and selects the code executor according to the task processing strategy; each task processing stage... Each has a corresponding task memory; Code executor, used from the task processing phase Retrieve tasks from the task storage, execute the code for the task processing stage based on the task processing stage data, and process the task data. Results The calculation results are then packaged into subsequent task processing stages. Task The data is sent to the corresponding task memory via a process communicator; each task processing stage has a corresponding computing process. Database, used to store the task processing stages in task processing. The result .

4. A multi-type computational collaborative method based on reinforcement learning and negative feedback control, based on the system described in any one of claims 1-3, characterized in that, The steps are as follows: 1) Obtain task information, including data source and task processing flow. Task processing code at each stage of the task processing flow, and task processing delay constraints. ; 2) Based on the task storage status, code execution status, and inter-process communication status, obtain the initial resource context. ; 3) Based on the initial resource context and task information, establish an optimization model to obtain the task processing strategy. The task processing strategy includes executing each task processing stage. The code executor and execution parameters used when processing task code ; 4) Based on the task processing strategy Task processing phase The task processing code processes task data and generates results for the task processing phase. and subsequent task processing phase The task; 5) Detect resource context and obtain the historical sequence of resource context. Simultaneously analyze the results of all task processing stages. Detect the task context and obtain the task context history sequence. The obtained resource context history sequence is merged with the task context history sequence to obtain a context analysis report. ; 6) By period By using reinforcement learning neural networks to process the context analysis report and weighted bias, PID hyperparameters are obtained. ; 7) By period Calculate weighted bias Based on weighted bias Get updated task processing strategy ; 8) Update task processing strategy Execute subsequent task processing stage The task processing code generates the results of subsequent task processing stages. and subsequent task processing phases The task, among which ; 9) Repeat steps 5) to 8) until the task is completed.

5. The multi-type computational collaborative method based on reinforcement learning and negative feedback control according to claim 4, characterized in that, Step 2) specifically includes: The initial resource context is obtained by detecting task storage, code execution, and inter-process communication. The initial resource context includes, but is not limited to, the load condition of the task storage. and task consumption Average computation time of code executor Memory usage of the code executor Load status of process communicator .

6. The multi-type computational collaborative method based on reinforcement learning and negative feedback control according to claim 5, characterized in that, Step 3) specifically includes: 31) Based on the initial resource context and task information, establish a constrained optimization model, with the task processing strategy as the optimization variable. It includes the execution of each task processing stage. The code executor used when processing task code and execution parameters Optimization model The expression is as follows: ； ； ； in, For the index of the code executor, For task phase set; Indicates the task processing stage Is it by the first Each code executor executes, if the task processing phase... By the If a code executor executes, then ,otherwise ; Constraints on task processing latency; Indicates the initial resource context; The precision function representing the task processing strategy is specifically expressed as follows: ， In the formula, Indicates the task processing stage The computational parameters and initial resource context affect the task processing phase. The functional relationship for processing precision. Indicates the task processing stage The computational parameters and initial resource context affect the task processing phase. Functional relationship for processing precision; The delay function representing the task processing strategy is specifically expressed as follows: ， In the formula, Indicates the task processing stage The computational parameters and initial resource context affect the task processing phase. The functional relationship for handling delay. Indicates the task processing stage The computational parameters and initial resource context affect the task processing phase. The functional relationship for processing precision. Indicates the task processing stage The scaling factor of the calculation parameters on the average processing latency; 32) Solve the optimization model using the enumeration method to obtain the task processing strategy. Specifically, this involves iterating through the various parameter combinations of the task processing strategy and selecting candidate task processing strategies. and initial resource context Substituting into the time delay function yields the time delay. If satisfied Then, substitute the parameter combination into the precision function to obtain the precision. Select The largest alternative task processing strategy, as the task processing strategy .

7. The multi-type computational collaborative method based on reinforcement learning and negative feedback control according to claim 6, characterized in that, Step 4) specifically includes: 41) According to the task processing strategy Calculation parameters in Sensor data is quantized according to specific parameters and sampling rate Perform sampling and quantization to obtain task data. And encapsulate it with task information into a task The task includes a task processing phase. Task data Task processing flow and task processing strategies ; 42) Utilizing the task processing phase Corresponding task memory Save the packaged task ; 43) According to the task processing strategy Select the code executor, and the code executor will determine the appropriate code executor based on the given code. Loading task processing phase The task processing code executes the task processing code using the calculation parameters in the task processing strategy, and processes the task data to obtain the processing results. The data is then saved to the database, and subsequent task processing stages are encapsulated. Task ; 44) The task Distributed to task processing stage The corresponding task storage.

8. The multi-type computational collaborative method based on reinforcement learning and negative feedback control according to claim 7, characterized in that, Step 5) specifically includes: 51) Periodically check resource conditions, including but not limited to the load status of task memory. and task consumption Average computation time of code executor Memory usage of the code executor Load status of process communicator Preserve the resource context for each detection Forming a historical sequence of resource contexts ; 52) Periodically detect task scenarios and analyze the task processing results stored in the database. Processing results Sum of winning bid data values and the dimensions of vector data values To form a task context Preserve the task context of each analysis to form a task context history sequence. ,Will Actual processing latency of tasks Extract and merge to form the actual processing delay sequence of the task. ; 53) Merge the resource context history sequence and the task context history sequence to obtain the context analysis report. .

Images