Heterogeneous computing resource adaptive configuration and allocation method and device, and storage medium

Abstract

The invention discloses a heterogeneous computing resource adaptive configuration and allocation method and device, and a storage medium. The method comprises the following steps: a video source device sends a previous video segment to a server; the server performs reasoning on a previous video segment to obtain historical reasoning time delay; the server determines the parameter configuration of the next video segment according to the historical network broadband, the maximum tolerable time delay, the queue backlog and the historical reasoning time delay, and allocates heterogeneous computing resources to the received video segment; the server feeds back the parameter configuration of the next video segment to the terminal; and the terminal encodes the video segments according to the video segment parameter configuration fed back by the server and sends the encoded video segments to the server. By adopting the technical scheme provided by the invention, the reasoning quality of the multi-channel video stream is maximized by dynamically adjusting the video block configuration and heterogeneous resource allocation strategy under the conditions of network dynamic fluctuation and limited edge server resources.

Description

Technical Field

[0001] The present invention belongs to the field of multimedia communication technology, and in particular relates to a method and device for adaptively configuring and allocating heterogeneous computing resources, and a storage medium. Background Art

[0002] In recent years, the rapid development of deep learning technology has promoted major breakthroughs in computer vision technology (such as target detection and classification). Real-time video analysis technology based on deep neural network (DNN) has been widely used in public security, industrial automation and other scenarios. DNN model reasoning consumes a lot of memory and computing resources. However, in actual scenarios, the computing and memory resources of video source devices (such as network cameras, drones, etc.) are limited, and local real-time video analysis cannot meet its performance requirements. The cloud computing-based video analysis framework sends the video stream generated by the video source device to the remote cloud with abundant computing resources for DNN model reasoning, but the long return link delay and random network fluctuations make it impossible to apply to scenarios with high real-time requirements. Real-time video analysis based on edge computing uses servers at the edge of the network to perform DNN model reasoning, which can significantly reduce video transmission delay. Therefore, the real-time video analysis framework based on edge computing has become a research hotspot. Real-time video analysis based on edge computing mainly consists of video transmission process and DNN model reasoning process. Under dynamic network conditions, video stream configuration has a direct impact on transmission latency, so dynamic adaptive configuration is widely adopted to reduce transmission latency. However, changes in video stream configuration (such as resolution, frame rate, video quality, etc.) also have a significant impact on the DNN reasoning process. On the other hand, GPUs have been widely used to accelerate DNN model reasoning, so in the reasoning process, video configuration and heterogeneous resources such as CPU, GPU, and memory jointly affect the reasoning speed. In order to ensure end-to-end latency and improve the overall quality of inference (QoI) of the system, the video stream configuration selection and the heterogeneous resource allocation strategy of ES should be considered jointly. In the multi-video stream analysis scenario, the differentiated end-to-end latency requirements will cause fierce competition for heterogeneous system resources, resulting in huge challenges in designing joint configuration selection of multiple video streams and heterogeneous computing resource allocation strategies.

[0003] In the edge computing-based multi-video stream real-time analysis framework, the typical transmission pipeline is that multiple EDs continuously capture and encode the original video images, and then send them to the edge server (ES) through a dynamic and constrained network. The ES decodes the video and performs DNN model inference. Many solutions use dynamic video transmission protocols (such as DASH) to split the video stream into a series of video segments of the same length and dynamically select the configuration of the video segments (such as resolution, frame rate, quality, etc.) to adapt to changes in network bandwidth. This naturally leads to a trade-off between video analysis accuracy and latency. Choosing a higher configuration will increase accuracy but increase transmission latency, while choosing a worse configuration will reduce transmission latency but waste available bandwidth. In addition, in the actual transmission process, there is inevitably a sending queue, making the adaptive configuration selection problem a Markov decision process. At the same time, according to the actual wireless communication system parameters, the channel coherence time is usually tens of milliseconds, which is much shorter than the video segment transmission time, which is usually hundreds of milliseconds. Since the actual network bandwidth is difficult to accurately predict, the video segment bit rate and network bandwidth cannot be accurately matched, resulting in the video segments sent by EDs arriving at the ES at non-uniform time intervals. Therefore, it is necessary to model the problem in combination with the actual time scale to achieve better performance in practice.

[0004] Unlike traditional video transmission, which focuses more on the experience of watching videos with human eyes, the results of real-time video analysis are often used to assist decision-making or trigger alarms. The former has a certain tolerance for latency and pays more attention to video quality, while real-time video analysis has more stringent requirements on end-to-end latency, so it is necessary to carefully consider the latency of each part of the video analysis pipeline. In addition to transmission latency, ES computing latency is also an important source of video analysis. DNN-based video reasoning usually includes three stages: preprocessing, model reasoning, and postprocessing. It is worth noting that video preprocessing and postprocessing are usually CPU-intensive calculations, while the DNN model reasoning process is GPU-intensive calculations. Therefore, DNN reasoning for each video segment requires heterogeneous computing resources to be coordinated to complete. In addition, DNN model reasoning is also a memory-intensive task, which requires a large amount of GPU memory to temporarily store model parameters, input data, intermediate results of forward propagation, and other data. At the same time, the batch size of the input neural network during DNN reasoning is also an important parameter affecting the reasoning speed. The batch size represents the number of images input into the model at one time. Due to the parallel design of the GPU, parallel reasoning of multiple frames can make full use of the GPU's computing power, thereby reducing reasoning latency. On the other hand, as the batch size increases, the GPU memory usage of the reasoning process increases. Due to the limitation of physical memory size, although a larger batch size can reduce latency, it may cause queuing latency. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method and device for adaptively configuring and allocating heterogeneous computing resources, and a storage medium, which can maximize the inference quality of multiple video streams by dynamically adjusting the video block configuration and heterogeneous resource allocation strategy under the conditions of dynamic network fluctuations and limited edge server resources.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] A method for adaptively configuring and allocating heterogeneous computing resources, comprising:

[0008] Step S1, the video source device sends the last video segment to the server;

[0009] Step S2: The server performs inference on the previous video segment to obtain a historical inference delay;

[0010] Step S3: The server determines the parameter configuration of the next video segment according to the historical inference delay and allocates heterogeneous computing resources to the received video segment;

[0011] Step S4: the server feeds back the parameter configuration of the next video segment to the terminal;

[0012] Step S5: The terminal encodes the video segment according to the video segment parameter configuration fed back by the server and sends the encoding to the server.

[0013] Preferably, in step S3, the server determines the parameter configuration of the next video segment and allocates heterogeneous computing resources to the received video segment according to historical network bandwidth, maximum tolerable delay, queue backlog, and historical reasoning delay through event-triggered decision making and dual-queue model SMDP.

[0014] Preferably, in step S3, the JCCGB strategy based on diffusion strategy reinforcement learning repeatedly trains the SMDP with the environment to obtain an ideal scheduling strategy.

[0015] The present invention also provides a heterogeneous computing resource adaptive configuration and allocation device, comprising:

[0016] The first processing module is used for the video source device to send the last video segment to the server;

[0017] The second processing module is used for the server to infer the previous video segment and obtain the historical inference delay;

[0018] The third processing module is used for the server to determine the parameter configuration of the next video segment according to the historical reasoning delay and allocate heterogeneous computing resources to the received video segment;

[0019] A fourth processing module, configured for the server to feed back parameter configuration of the next video segment to the terminal;

[0020] The fifth processing module is used for the terminal to encode the video segment according to the video segment parameter configuration fed back by the server and send the encoding to the server.

[0021] Preferably, the server determines the parameter configuration of the next video segment and allocates heterogeneous computing resources to the received video segments based on historical network bandwidth, maximum tolerable delay, queue backlog, and historical reasoning delay through event-triggered decision making and dual-queue model SMDP.

[0022] As a preferred method, the JCCGB strategy based on diffusion strategy reinforcement learning repeatedly trains SMDP with the environment to obtain an ideal scheduling strategy.

[0023] The present invention also provides a storage medium, on which a computer program is stored, and the computer program executes the method for adaptively configuring and allocating heterogeneous computing resources when running.

[0024] The present invention is based on the edge server video analysis scenario of heterogeneous computing resources, taking into account the actual time scale, and constructing an event-triggered decision-making problem model. In addition, the present invention adopts a diffusion strategy-based reinforcement learning method for adaptive video configuration selection and heterogeneous resource allocation. For the multi-video stream heterogeneous edge server video analysis scenario, the present invention can adaptively make video configuration adaptation and heterogeneous resource (including CPU, GPU, and memory) scheduling decisions according to dynamically changing and limited network conditions, and can maximize the long-term average inference quality (QoI) of multiple video streams under the conditions of end-to-end time constraints given a multi-video stream inference model, thereby improving system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0026] Figure 1 A flowchart of a method for adaptively configuring and allocating heterogeneous computing resources according to an embodiment of the present invention;

[0027] Figure 2 Architecture diagram for real-time analysis of multiple video streams;

[0028] Figure 3The following are test diagrams showing the effect of inference parameter settings on inference complexity and resource usage when using GPU for DNN model inference. (a) shows the change of GPU memory usage and inference latency of ResNet18 with input batch size when using desktop-level GPU RTX 2080Ti, and (b) shows the change of computational complexity of YOLOv5 model with input resolution and frame rate when using RTX3080.

[0029] Figure 4 It is a dual queue model and time-triggered decision making;

[0030] Figure 5 It is the forward process and reverse process of the diffusion process;

[0031] Figure 6 It is a JCCGB strategy training framework based on diffusion strategy reinforcement learning. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Embodiment 1:

[0035] The present invention provides a method for adaptively configuring and allocating heterogeneous computing resources to solve the problem of adaptive configuration adjustment and heterogeneous resource allocation in edge real-time video analysis systems. Figure 1 As shown, the following steps are included:

[0036] Step S1, the video source device sends the last video segment to the server;

[0037] Step S2: The server performs inference on the previous video segment to obtain a historical inference delay;

[0038] Step S3: The server determines the parameter configuration of the next video segment and allocates heterogeneous computing resources to the received video segment through event-triggered decision-making and dual-queue model SMDP according to historical network bandwidth, maximum tolerable delay, queue backlog, and historical reasoning delay; wherein, the JCCGB strategy based on diffusion strategy reinforcement learning repeatedly trains SMDP with the environment to obtain an ideal scheduling strategy;

[0039] Step S4: the server feeds back the parameter configuration of the next video segment to the terminal;

[0040] Step S5: The terminal encodes the video segment according to the video segment parameter configuration fed back by the server and sends it to the server.

[0041] like Figure 2 As shown in Figure 1, the video analysis system includes: a video source device, multiple wireless access points, and an edge computing server. The video source device continuously captures real-time video images and sends the video to the ES for video analysis through wireless communication methods such as 4G, 5G, and WiFi. EDs are represented as The computing resources of ES are represented as {N C ,f C ,N G ,f G}, where N C ,f C N is the number of ES CPU cores and the computing power of a single-core CPU (in FLOPS). G 、f G They are the number of ES GPU streaming multiprocessors (SMs) and the computing power of a single SM (in FLOPS). After receiving the video segment, ES needs to allocate a certain number of CPU, GPU, and memory resources to the video segment to perform preprocessing, model inference, and postprocessing operations. When ES performs DNN model inference on the video segment, the general preset is adopted: the video preprocessing and postprocessing process uses the CPU for calculation, and the DNN model inference uses the GPU for calculation.

[0042] In the real-time video stream analysis pipeline, the video stream is divided into L video segments of equal length and transmitted independently. All video segments generated by the video source device are represented as The length of each video segment is denoted as x. For a continuous video stream, L can be infinite. During the transmission process, after the current video segment is transmitted, the ES sends the configuration information of the next video segment to the corresponding video source device. The video source device then encodes and transmits the video block according to the configuration information. The options for the parameter configuration combination in this article are {resolution, frame rate, video quality}. The configuration of the lth video segment of the video stream is denoted as in is the resolution of the lth video segment, is the frame rate of the lth video segment, is the video quality of the lth video segment. Indicates video segment Adopt configuration The size of the encoded data (i.e., the size transmitted over the network). In actual wireless communication environments, the channel coherence time is much shorter than the video block transmission time. Therefore, the time slot is represented as {1,2,…,t,…,T}, assuming that the length of each time slot is τ and the channel state does not change within each time slot. The video segment transmission delay will depend on the data size and the network bandwidth in the future. The network bandwidth of the video source device in the tth time slot is represented as B m (t), if the video segment When transmission starts at the nth time slot, the transmission delay is It can be approximated as:

[0043]

[0044] Reasoning accuracy is another important indicator for real-time video stream analysis. However, the performance improvement between latency and accuracy is contradictory. The change of accuracy with the increase of bit rate is a concave function, that is, the improvement effect of configuration on accuracy gradually decreases, which provides opportunities for real-time video stream analysis, such as sacrificing less accuracy to reduce a large amount of transmission delay when network conditions are poor. In addition, the accuracy of model reasoning is not only affected by the configuration of the video segment itself, but also by other factors, such as the multiple implementation methods of the DNN model, the dynamic degree and complexity of the DNN model reasoning depth content, and it is difficult to build a unified accuracy model to characterize the impact of various parameter configurations on accuracy. Therefore, the embodiment of the present invention simplifies the accuracy model to a more general situation: 1) Each video stream uses a specific DNN model for reasoning and the model parameters and reasoning depth are determined in advance; 2) The video block size is used to represent the dynamic degree of video content change. This is because the video segment with more complex content and greater degree of change has a larger data size after encoding.

[0045] In the mainstream CPU-GPU heterogeneous computing platform, programming models such as CUDA and OpenCL all use the host-device architecture. Before kernel execution (for example, DNN model inference), GPU memory must be allocated for the task and the data must be transferred to the memory. A memory management method similar to that in is adopted. Since the model memory requirements are static and predictable, the DNN model parameters are fixedly loaded into the memory, and each time the inference task is executed, memory is dynamically allocated for the input, output, and intermediate inference results. It is worth noting that the choice of batch size will directly affect the GPU video memory usage and inference speed. Therefore, the allocation of video memory is indirectly controlled by controlling the batch size of the input. The batch size chosen during inference is expressed as The batch size cannot exceed the video segment The number of frames, expressed as:

[0046]

[0047] Among them, the definition To adopt The amount of CPU calculation required for configuration, To adopt The amount of GPU computing required during configuration, Assign the server to the video segment respectively The number of CPU cores and SMs, defined The input batch size of the model is The video memory usage at that time. You can get the video segment The computational delay

[0048]

[0049] in, The function is used to reflect the impact of batch size on the speed of the model inference process. It should be noted that as well as All of them are functions related to specific models and devices. In practice, they can be fitted using offline experimental regression methods. The desktop GPU RTX3080 is used to test the changes in ResNet18 video memory usage and inference latency with input batch size, and the changes in YOLOv5 model computational complexity with input resolution and frame rate. Figure 3 shown.

[0050] G max is the maximum available GPU memory of ES (i.e., the memory that can be used to store input, output, and intermediate inference results excluding model parameters), and the set of video segments that are performing model inference is represented as but The total GPU memory occupied by the video segments in the max , expressed as:

[0051]

[0052] At the same time, the number of allocated CPUs and GPUs cannot exceed the maximum number of available CPUs and GPUs in ES, so we can get:

[0053]

[0054] When a new video block arrives at ES but the available resources cannot meet the demand, the newly arrived video block will enter the ES's computing queue Z(t) and wait until enough resources are released. Definition For video segment The waiting time of the video segment Total analysis delay for:

[0055]

[0056] When receiving a video segment, the server will allocate computing resources for the video segment and select an appropriate inference batch size based on the end-to-end latency requirements and system status information of the corresponding video source device. Figure 4 As shown in Figure 1, due to the continuous changes in video configuration and network bandwidth, the time interval for video segments to arrive at the server will change dynamically, which causes the server's decision interval to change accordingly. Therefore, the decision process is modeled as SMDP. The video source device continuously captures new video segments. When the previous video segment is not transmitted within the next x time, the newly generated video segment will be placed in the sending queue H of the video source device. m (t) waiting to be sent. The video segment transmission completion is defined as an event, and the changes in the sending queue of the video source device and the computing queue of the server are triggered by this event. Specifically, when the video segment When the transmission is completed, the server will be triggered to make a video segment Encoding configuration strategy and video segment The computing resource and memory resource allocation strategy of . Assume that the time slot triggered by the kth event is The video segment index of the triggering event is l', and the size of the original video stream data with a duration of τ is D. Figure 4 As shown, H m The queue length of (t) can be expressed as:

[0057]

[0058] in,

[0059] DNN reasoning requires that the CPU, GPU, and memory resources meet the requirements at the same time before the task can be executed. After the reasoning is completed, the CPU, GPU, and memory resources will be released at the same time. Therefore, the CPU, GPU, and memory usage time are equal and equal to the total computing time. The queue Z(t) is divided into the CPU computing queue Z C (t), GPU computing queue Z G (t) and memory queue Z M (t), we can get Z C (t), Z G (t) and Z M (t) changes with time as:

[0060]

[0061]

[0062] as well as

[0063]

[0064] In a multi-video stream analysis system, the latency requirements of video stream analysis tasks are often different. For example, the video source used to monitor dangerous events requires that the analysis results and alarm information be obtained within a short latency. The maximum tolerable latency of the video stream in the system is expressed as T max In order to maximize the overall QoI of the system while ensuring latency, the server needs to dynamically select video segment configuration and allocate heterogeneous resources based on network dynamics, latency constraints of different video streams, and system resource status. Adopt configuration The accuracy of the analysis is expressed as The optimization target is determined as the long-term average QoI of the video segment, that is, the weighted end-to-end delay and accuracy:

[0065]

[0066] Among them, ι1 and ι2 are the weight coefficients of inference accuracy and end-to-end delay respectively. Belongs to SMDP, and the problem is transformed based on the SMDP model. SMDP can usually be expressed as in, represents the state space, is the action space, t is the decision period, P is the state transition probability, and R is the instantaneous reward. The details are as follows:

[0067] (1) State space In the constructed SMDP model, the state space can be expressed as:

[0068]

[0069] in, Indicates the historical information of network bandwidth within a certain period of time. For the dynamic content of the video, the video block size under the same configuration is used to reflect the dynamic of the video block. It should be noted that using the video segment size to reflect the dynamic of the video segment is only an example, and other methods of reflecting the dynamic are also applicable to the joint configuration adaptation and resource allocation strategy. Indicates the queue backlog in the system, including the send queues of all video source devices and the CPU and GPU resource occupation queue lengths on the ES side.

[0070] (2) Action space A: When an event is represented by a video block When triggered, the decision made by ES is:

[0071]

[0072] in, For video blocks Video configuration, Assigned to The number of CPU cores and GPUSMs, for Batch size decision during model inference. Since the number of frames in a video segment will change, a continuous variable in the range of (0, 1) is output and scaled to the corresponding integer batch size. The decision variables for the number of CPUs and GPUs are similar.

[0073] (3) Decision period t: When an event is triggered, a decision is made and then the environment state is transferred to the next state, but the interval between the environment state transfers is random. k ,k∈{0,1,2,…} represents the kth decision period, and the interval between two adjacent decisions is determined by the transition probability distribution.

[0074] (4) Transition probability P: The probability that the environment transitions from state s to the next state s' is expressed as p(s'|s,a). The transition time between two adjacent states is a random variable, expressed as p(t k+1 -t k ≤ζ,s'|s,a) means that when the environment state is s, the time interval of state transition after taking action a is less than or equal to ζ. By differentiating both sides with respect to ζ, we can obtain the probability density function that when the environment state is s, the time interval of state transition after taking action a is less than or equal to ζ: Then the expectation of ζ is: For the sake of simplicity, we will use to represent the expected value of ζ.

[0075] (5) Reward Function When the server takes action a in state s, the instantaneous reward is The reward function is designed according to the optimization goal. Since the queue status will indirectly affect the long-term average QoI of the system, a reward function related to the queue length and QoI is designed at the same time. The reward function related to the queue length is where c Q is the coefficient of queue length; the reward function related to the video segment QoI is Where ι3 is the timeout penalty coefficient;

[0076] As an implementation method of an embodiment of the present invention, the JCCGB strategy based on diffusion strategy reinforcement learning is as follows:

[0077] use as well as To represent the device sending queue and the server computing queue respectively, Represents the combined vector of the two, and defines the Lyapunov function as:

[0078]

[0079] L(Θ(t k )) represents the queue backlog level of the dual-queue system, L(Θ(t k )) is larger, the more serious the backlog is and the worse the stability of the queue is. Introducing Lyapunov drift Δ(Θ(t k )) to measure the stability change of the dual queue system:

[0080]

[0081] Since we want to maximize the long-term average QoI of multiple video streams, the Lyapunov drift plus penalty function can be written as:

[0082]

[0083] Where μ is a factor that weighs queue backlog and video stream QoI. The maximum interval between two adjacent events is defined as ζ max , The maximum calculation time is According to Lyapunov optimization theory, the upper bound of the Lyapunov drift plus penalty function is:

[0084]

[0085] in:

[0086] is a constant, the optimization problem can be transformed into:

[0087]

[0088] Due to the problem It is a sequential decision problem and the decision variables are coupled with each other, so deep reinforcement learning is used to solve this problem. Reinforcement learning can be generally divided into value-based reinforcement learning and policy-based reinforcement learning. The former continuously corrects the value function Q(s,a) of the action through value iteration, which often cannot cope with the dimensionality disaster caused by the huge action and state space. Policy-based reinforcement learning directly learns the policy function and optimizes the policy network parameters through gradient update. When making decisions, the probability of the action is directly obtained according to the state, avoiding the dimensionality disaster. However, traditional policy networks widely use unimodal Gaussian distribution when fitting policies, which makes it impossible for the intelligent agent to learn multimodal policies, and the lack of fitting ability for complex distributions will eventually cause the policy to fall into local optimality or unstable training. Recently, the policy representation method based on the diffusion model has been introduced into reinforcement learning, and it has shown better performance than the parameter policy function in a multimodal environment. Therefore, the reinforcement learning method DIPO based on the diffusion model is used to train the optimal JCCGB policy.

[0089] like Figure 5 As shown in the figure, the diffusion strategy model includes a forward process and a reverse process. When a state s is given, the forward process generates a series of Starting action Its stochastic differential equation can be expressed as:

[0090]

[0091] use Representing the evolution distribution of the random process (17), we can get the conditional distribution Obeys Gaussian distribution

[39] :

[0092]

[0093] This means that the forward process (17) transforms the policy π(·|s) into Gaussian noise Therefore, the noise can be restored to the random strategy π(·|s) through the inverse process, that is, the approximate original strategy distribution is obtained according to the subsequent random process

[0094]

[0095] Among them, p t (·)for The probability density function of . According to

[39] , when When the reverse process A time-reversed version of the forward process have the same distribution, that is, for t = 0, 1, 2, ..., T, if but But the score function in (19) It cannot be obtained accurately, and an estimation function is used to approximate the score function under a given state s. You can get:

[0096]

[0097] in, is a set of function approximators (such as neural networks). The inverse process is discretized by exponential integration

[40] . Assuming that the time length of the inverse process ranges from [0, T] and K represents the total discrete step length, we can get Therefore, the discrete version of the inverse process can be expressed as:

[0098]

[0099] The initial distribution of this random process is By changing (19) to [t k ,t k+1 ], we can get the exact solution of the random process (20) as:

[0100]

[0101] Therefore, the strategy π(·|s) can be approximated with the help of the score function. Algorithm 1 shows the detailed steps.

[0102]

[0103] Since the score function is unknown, it is necessary to continuously learn the score function in the process of updating the strategy. Using denoising score matching, the problem can be expressed as:

[0104]

[0105] Among them, ω(t), t∈[0,T] is a positive weighted function, represents the transfer kernel of the forward process (15), express The expectation of φ is the parameter to be learned; the optimization objective is rewritten as:

[0106]

[0107] in, represents uniform distribution, and When making action decisions, we use ∈ φ Estimates of alternative scoring functions After algebraic transformation, the state s Same as previous action Get the next action

[0108]

[0109] in, The agent performs reasoning along the time sequence of k=0,1,…,K-1 and outputs the final action

[0110] Unlike the parameter strategy based on neural networks, the diffusion strategy can only fit the strategy by approximating an SDE, and cannot directly improve the strategy by gradient descent. The diffusion strategy can fit the distribution of a random process in an existing data set. Therefore, Figure 6 As shown in Figure 1, the “action gradient” is used to improve the state-action pair (s, a) in the data set to obtain an improved data set, that is, the original (s, a) is replaced with a state-action pair (s', a') with a higher reward value, and finally the improved diffusion strategy π' (π' ≥ π) is obtained using the improved data set. The specific steps are:

[0111] 1) The reward value is regarded as a function of the action to characterize the impact of the action on the reward value, that is,

[0112] 2) Update the action value according to the action gradient:

[0113]

[0114] Where λ is the step size. To implement (25) from the samples of the dataset, we need to fit Q π (s,a). Similar to the DQN algorithm, a parameterized neural network is used to fit Q π (s, a) and introduce the double Q learning technique. The Bellman residual that needs to be minimized during the training process is:

[0115]

[0116] Finally, for all actions in the dataset Will replace all according to the following rules

[0117]

[0118] The specific algorithm flow is shown in Algorithm 2.

[0119]

[0120]

[0121] Embodiment 2:

[0122] The embodiment of the present invention further provides a heterogeneous computing resource adaptive configuration and allocation device, comprising:

[0123] The first processing module is used for the video source device to send the last video segment to the server;

[0124] The second processing module is used for the server to infer the previous video segment and obtain the historical inference delay;

[0125] The third processing module is used for the server to determine the parameter configuration of the next video segment according to the historical reasoning delay and allocate heterogeneous computing resources to the received video segment;

[0126] A fourth processing module, configured for the server to feed back parameter configuration of the next video segment to the terminal;

[0127] The fifth processing module is used for the terminal to encode the video segment according to the video segment parameter configuration fed back by the server and send the encoding to the server.

[0128] As an implementation method of an embodiment of the present invention, the server determines the parameter configuration of the next video segment and allocates heterogeneous computing resources to the received video segments based on historical network bandwidth, maximum tolerable delay, queue backlog, and historical reasoning delay through event-triggered decision-making and dual-queue model SMDP.

[0129] As an implementation method of the embodiment of the present invention, the JCCGB strategy based on diffusion strategy reinforcement learning repeatedly trains SMDP with the environment to obtain an ideal scheduling strategy.

[0130] Embodiment 3:

[0131] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the method for adaptively configuring and allocating heterogeneous computing resources when running.

[0132] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for adaptively configuring and allocating heterogeneous computing resources, characterized in that: include: Step S1, the video source device sends the last video segment to the server; Step S2: The server performs inference on the previous video segment to obtain a historical inference delay; Step S3: The server determines the parameter configuration of the next video segment according to the historical inference delay and allocates heterogeneous computing resources to the received video segment; Step S4: the server feeds back the parameter configuration of the next video segment to the terminal; Step S5: The terminal encodes the video segment according to the video segment parameter configuration fed back by the server and sends the encoding to the server.

2. The method for adaptively configuring and allocating heterogeneous computing resources according to claim 1, characterized in that: In step S3, the server determines the parameter configuration of the next video segment and allocates heterogeneous computing resources to the received video segment based on historical network bandwidth, maximum tolerable delay, queue backlog, and historical reasoning delay through event-triggered decision making and dual-queue model SMDP.

3. The method for adaptively configuring and allocating heterogeneous computing resources according to claim 2, characterized in that: In step S3, the JCCGB strategy based on diffusion strategy reinforcement learning repeatedly trains the SMDP with the environment to obtain an ideal scheduling strategy.

4. A device for adaptively configuring and allocating heterogeneous computing resources, characterized in that: include: The first processing module is used for the video source device to send the last video segment to the server; The second processing module is used for the server to infer the previous video segment and obtain the historical inference delay; The third processing module is used for the server to determine the parameter configuration of the next video segment according to the historical reasoning delay and allocate heterogeneous computing resources to the received video segment; A fourth processing module, configured for the server to feed back parameter configuration of the next video segment to the terminal; The fifth processing module is used for the terminal to encode the video segment according to the video segment parameter configuration fed back by the server and send the encoding to the server.

5. The device for adaptively configuring and allocating heterogeneous computing resources according to claim 4, characterized in that: The server uses event-triggered decision-making and the dual-queue model SMDP to determine the parameter configuration of the next video segment based on historical network bandwidth, maximum tolerable delay, queue backlog, and historical inference delay, and allocates heterogeneous computing resources to the received video segments.

6. The heterogeneous computing resource adaptive configuration and allocation device according to claim 5, characterized in that: The JCCGB strategy based on diffusion strategy reinforcement learning repeatedly trains SMDP with the environment to obtain an ideal scheduling strategy.

7. A storage medium, characterized in that: The storage medium stores a computer program, which, when running, executes the method for adaptively configuring and allocating heterogeneous computing resources as described in any one of claims 1 to 3.

Images