Task scheduling method and device for continuously learning edge surveillance video stream, equipment and medium

Abstract

The invention relates to the technical field of video processing platforms, and discloses a task scheduling method, device and equipment for continuously learning an edge surveillance video stream and a medium, and the method comprises the steps: distributing an initial memory resource into a task according to a uniform distribution strategy; evaluating the resource utilization rate of the task, and evaluating the quality of the current allocation strategy based on the resource utilization rate of the task; dynamically adjusting and allocating target memory resources based on the resource utilization rate of the task; and dynamically adjusting the configuration of the task based on the target memory resource. In order to maximize the global resource utilization rate and the global model accuracy, in each time window, fewer memory resources are allocated to the tasks with too low resource utilization rate, and task configuration levels are reduced for the tasks with too high accuracy (the higher the task configuration levels are, the higher the accuracy is, but the more the consumed resources are); more memory resources are allocated to the tasks with the overhigh resource utilization rate, and task configuration gears are improved for the tasks with the overlow accuracy (the lower the task configuration gears are, the lower the resource consumption is, but the accuracy is reduced).

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a task scheduling method, device, equipment and medium for continuous learning from edge monitoring video streams. Background Art

[0002] With the rapid growth of data volume and privacy protection issues, edge computing has become more and more widely used in video security, industrial Internet, smart agriculture, smart cities and other fields. At the same time, with the development of artificial intelligence, the demand for intelligent massive surveillance cameras is increasing, and it is necessary to deploy some AI models for the cameras at the edge to realize the intelligence of the camera video stream.

[0003] The distribution of data captured by the camera is constantly changing. For example, the scenes and objects captured by the camera will change over time, for example, the density of people and objects in the surveillance video will change over time. It is impossible to enumerate all scenes and objects when training the model, so "continuous model retraining" is needed to solve this problem. However, it is a huge challenge to complete the retraining task and the video inference task on the edge server at the same time. If more resources are allocated to the retraining task while the overall resources remain unchanged, the overall GPU utilization will be affected. If more resources are allocated to the inference task, retraining will take a long time, which will affect the accuracy of the inference model. The traditional scheduling method is to evenly distribute resources, which cannot achieve the best results. Summary of the invention

[0004] The present invention provides a task scheduling method, device, computer equipment and medium for continuous learning from edge monitoring video streams to solve the technical problem that it is a very big challenge to complete retraining tasks and video inference tasks on edge servers at the same time.

[0005] In a first aspect, a task scheduling method for continuous learning from edge monitoring video streams is provided, comprising:

[0006] Allocate initial memory resources to tasks according to the uniform allocation strategy;

[0007] Evaluate the resource utilization of tasks and evaluate the quality of the current allocation strategy based on the resource utilization of tasks;

[0008] Dynamically adjust the memory resources allocated as the target based on the resource utilization of the task;

[0009] Dynamically adjust the configuration of tasks based on target memory resources.

[0010] In a second aspect, a task scheduling device for continuously learning from an edge monitoring video stream is provided, comprising:

[0011] The initial allocation module is used to allocate initial memory resources to tasks according to the uniform allocation strategy;

[0012] The evaluation module is used to evaluate the resource utilization of the task and evaluate the quality of the current allocation strategy based on the resource utilization of the task;

[0013] A dynamic resource adjustment module is used to dynamically adjust the target memory resources based on the resource utilization of the task;

[0014] The dynamic configuration adjustment module is used to dynamically adjust the configuration of tasks based on target memory resources.

[0015] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the task scheduling method for continuous learning from edge monitoring video streams are implemented.

[0016] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the task scheduling method for continuous learning from edge monitoring video streams are implemented.

[0017] In the scheme implemented by the above-mentioned task scheduling method, device, computer equipment and storage medium for continuous learning from edge monitoring video streams, initial memory resources can be allocated to tasks according to a uniform allocation strategy; the resource utilization of the task is evaluated, and the quality of the current allocation strategy is evaluated based on the resource utilization of the task; the allocation is dynamically adjusted to the target memory resource based on the resource utilization of the task; the configuration of the task is dynamically adjusted based on the target memory resource. In order to maximize the global resource utilization and global model accuracy, in each time window, fewer memory resources are allocated to tasks with too low resource utilization, and the task configuration gear is reduced for tasks with too high accuracy (the higher the task configuration gear, the higher the accuracy, but the more resources consumed), more memory resources are allocated to tasks with too high resource utilization, and the task configuration gear is increased for tasks with too low accuracy (the lower the task configuration gear, the less resources are consumed, but the accuracy will be reduced). After the improvement, the global resource utilization and global model accuracy can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0019] Figure 1Schematic diagram of an application environment of a task scheduling method for continuous learning from edge monitoring video streams in one embodiment of the present invention;

[0020] Figure 2 1 is a flow chart of a task scheduling method for continuous learning from edge monitoring video streams in one embodiment of the present invention;

[0021] Figure 3 It is a structural diagram of a task scheduling device for continuously learning from edge monitoring video streams in one embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0023] Figure 5 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] The task scheduling method for continuous learning from edge monitoring video streams provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In an application environment, the client allocates initial memory resources to tasks according to a uniform allocation strategy; evaluates the resource utilization of the task, and evaluates the quality of the current allocation strategy based on the resource utilization of the task; dynamically adjusts the allocation to the target memory resources based on the resource utilization of the task; and dynamically adjusts the configuration of the task based on the target memory resources. In order to maximize the global resource utilization and global model accuracy, in each time window, fewer memory resources are allocated to tasks with too low resource utilization, and tasks with too high accuracy have their task configuration gear lowered (the higher the task configuration gear, the higher the accuracy, but the more resources consumed), more memory resources are allocated to tasks with too high resource utilization, and the task configuration gear is increased for tasks with too low accuracy (the lower the task configuration gear, the less resources are consumed, but the accuracy will be reduced). After the improvement, the global resource utilization and global model accuracy can be improved. The present invention is described in detail below through specific embodiments.

[0026] See also Figure 2 As shown, Figure 2 A flowchart of a task scheduling method for continuous learning from edge monitoring video streams provided by an embodiment of the present invention includes the following steps:

[0027] S10: Allocate initial memory resources to tasks according to the uniform allocation strategy;

[0028] In this embodiment, in order to maximize the global memory resource utilization and the global model accuracy, in each time window, fewer memory resources are allocated to tasks with too low resource utilization, and the task configuration gear is lowered for tasks with too high accuracy (the higher the task configuration gear, the higher the accuracy, but the more resources consumed), more memory resources are allocated to tasks with too high resource utilization, and the task configuration gear is increased for tasks with too low accuracy (the lower the task configuration gear, the less resources are consumed, but the accuracy will be reduced).

[0029] The main computing unit in the GPU architecture (memory architecture), capable of processing various mathematical and logical operations. Memory system: includes L1, L2 cache and shared memory, etc., used to store data and instructions to reduce the latency of the GPU system accessing the main memory. Cache and cache line: used to improve the memory access efficiency of the GPU.

[0030] It should be noted that the GPU is asynchronous, and generally the CPU does not run after issuing a command. Issuing kernel instructions is very fast, and in theory the CPU can do its own thing after the task is given to the GPU. Where there is asynchrony, there is synchronization. The CPU thread that relies on the GPU processing results needs to synchronize operations to wait for the current GPU to complete the processing before proceeding to the next step. However, the above GPU utilization rate is not high. When the GPU finishes executing, the CPU actually does nothing. The data copy between the GPU and the CPU takes time. After the GPU is processed, the CPU will do some subsequent operations, and finally process the next image. Coupled with some language-level overhead, the GPU utilization rate is very low.

[0031] In fact, throwing all the tasks to the GPU at once, because the GPU is asynchronous, the CPU can pass the task to it and obtain the data processed by the GPU through synchronization. The kernel tasks passed to it will be queued by the GPU to be executed one by one, and the utilization rate of the GPU will be higher at this time, after all, it is relatively less idle. This is the most basic general method to keep the GPU from being idle. Try to do the following: reduce unnecessary CPU-GPU synchronization, and execute the kernel when copying data between the CPU and GPU. For many tasks, if the performance can be improved through asynchrony, try to make them asynchronous as much as possible, and make use of all the resources in the GPU.

[0032] In another embodiment, when getting a graphics card or computing card, people who are sensitive to computing power will generally care more about the computing resources of this card, such as A100. A100 has 108 SMs (stream processing units) and a total of 221,184 threads. Each SM has its own registers, shared video memory, shared L2 cache, etc. What needs to be done is to make full use of these resources as much as possible. If only one of the threads is used at a certain moment, and then it is called all the time (the kernel keeps running), at this time, although the GPU utilization checked by the nvidia-smi command is 100%, in fact, the usage for the graphics card is super low. For example, this single-block single-thread kernel will theoretically only occupy one SM. But the GPU utilization obtained by the nvidia-smi command is 100%, which is something to note.

[0033] How the GPU works: First, divide the data into blocks of equal size so that each block can be run independently in parallel. Because the blocks are independent, they can be scheduled in any order and at any time. This gives the hardware as much freedom as possible to pack things efficiently. One guarantee for each block is that it has a certain number of threads, and they are all guaranteed to run simultaneously on the same SM. Next, we will discuss how to use the threads in these blocks. There is a specific number of threads in each block, and they are all assigned to the same SM. The threads in the SM are divided into a certain number of groups, which are called warps. Each warp contains a specific number of threads (usually 32), and these threads are executed in the same clock cycle. This is one reason why a large number of threads are used - to keep all SMs and threads in the GPU as busy as possible.

[0034] When all blocks running on an SM are full, no new blocks will be added to the SM until the threads on the SM start to finish their work and free up some space. In CUDA, each streaming multiprocessor (SM) can process multiple blocks at the same time, the specific number depends on the size of the block, and can process up to 32. The GPU will continue to schedule blocks to each SM until all blocks are processed. This is called "occupancy", which is an important GPU performance indicator.

[0035] How to write code to take advantage of these GPU resources. CUDA provides a programming model called kernel functions, which are codes executed on the GPU. These functions can be called on the host to start computations on the GPU. An important feature of the kernel function is that it can be called multiple times, each time with a different block of data. This allows the GPU to utilize its parallel computing resources as efficiently as possible.

[0036] In order for the code to run on the GPU, some special keywords and functions need to be used in the host code so that the CUDA compiler can convert it into code that the GPU can execute. Pay attention to the corresponding places of the relevant hardware resources and the code. The kernel calculates the distance between multiple points to demonstrate parallel computing. Shared memory and registers are used in the program. Shared memory allows threads within a block to communicate, while registers provide the necessary working space for each thread. On the GPU, the purpose of registers is different from that of the CPU. They provide a large amount of immediately accessible data space, which is especially important for performing complex mathematical operations.

[0037] Each SM uses two-thirds more threads than before, which is almost a free performance gain. This "Tetris" problem has an additional dimension, but the hardware will solve this problem and let the core handle it.

[0038] As mentioned before, all tasks can be thrown to the GPU at once. In other words, the algorithm to be calculated can be submitted to the asynchronous work queue in advance (why in advance? It means that the task has been sent out and kernellaunched, but has not actually been executed). This queue is generally called a stream. If it is not set, the task will be in the default stream.

[0039] The GPU can have many streams, and the number of streams used varies depending on the task. In normal use, you can make full use of the characteristics of streams. One task is to use a neural network to process a picture of a flower, which involves copying a flower or flower data to the GPU's memory, then processing and processing it on the GPU, and finally copying the result back.

[0040] Because processing is not just one step, but multiple steps, this provides an opportunity for overlap, which means that independent work can be distributed into independent streams, and processing the second flower does not actually require waiting for the first flower to complete. The copy operation (copying data to the GPU), that is, data transfer, can be performed simultaneously with the calculation (kernel execution) on different hardware paths, so that the hardware resources can be efficiently utilized. Keep the GPU busy to the maximum extent. In fact, this is very similar to the Tetris mentioned earlier. The blocks here refer to various calculations and memory copy tasks, and then as many tasks as possible are filled into the available resources.

[0041] Now that we have kept the GPU as busy as possible and utilized its resources as much as possible, the next step is the hard part: using it efficiently. If the number of threads that the GPU can provide is not used, then it is obviously a waste and the GPU is not being used well. From a memory perspective, if we only consider the data input rate, although this is not completely representative of the actual situation (because there will be some data reuse), the data transfer rate of 1.5TB per second is a good performance indicator. This is equivalent to processing 194 billion double-precision floating-point values ​​per second, divided by 8, and we get a peak performance of 194GFLOPS. The GPU is now running at about the same speed as the iPhone 11. This example shows that when doing performance evaluation, we should fully consider not only the data transfer rate, but also the program’s ability to actually utilize this data, and an example of how memory bandwidth can become a performance bottleneck.

[0042] Let's look at how memory works. In a DRAM memory chip, each storage cell is a capacitor that stores one bit of information - a charged state represents 1, and no power represents 0. These capacitors are connected to the circuit through transistors, and the switching of the transistors controls whether the charge state of the capacitor can be read. The storage cells composed of these capacitors and transistors are connected in a two-dimensional matrix. This layout makes DRAM a random access memory (RAM). Unlike tape storage with sequential access, DRAM can access data through index addressing of rows and columns. When data is read, the selected row is first activated, and the state of all cells in the row is copied to the sense amplifier. The sense amplifier converts the weak charge in the capacitor into a well-defined voltage signal that can be easily read, which is convenient for reading. However, the charge in the capacitor is consumed during the reading process, resulting in the loss of the original data. Next, the column access is performed. Remember that this is a 2D matrix, and the row must be read before the column. Compared with reading data from the memory cell, it can read the data stored in the sense amplifier. This is faster than the row because the amplifier produces a strong, clear signal, and the reading does not destroy the data. Since adjacent memory locations are often read, there is also a continuous read mode in which a single request returns multiple data words. This is very important because it means I don't have to pay the cost of column requests over and over again. This is also one of the common optimization points in CUDA programming.

[0043] DRAM's burst mode: multiple words of data can be returned in a single request, which reduces the cost of column requests. The design of this read pattern reflects the impact of the physical storage structure - the charge and discharge characteristics of DRAM cells. Since row changes are costly, optimizing data access patterns to read adjacent data values ​​from the same row as much as possible can significantly improve read efficiency. This physical and technical detail is critical to understanding and optimizing memory access patterns, especially when performing memory-intensive operations such as CUDA programming.

[0044] But when it is time to access another row, the data in the current amplifier must be written back to the DRAM row to avoid data corruption (remember that the row has been depleted by the time the data is moved to the amplifier, so it must be rewritten to avoid memory corruption). Read eight byte values, reading data by increasing the step size. A step size of 8 means reading adjacent values, a step size of 16 means reading every other value, and so on, until the step size is 1000.

[0045] The data access pattern is indeed very important. This optimization premise is also common in CPU optimization, but it is particularly serious on GPUs because memory latency is longer. For example, a 2D array, which is a common image (h,w,1). It is usually laid out in a row-major format, where adjacent memory locations increase in the X direction. Here, if the step size used for reading increases each time, the performance will decrease. Similarly, if it is changed to access along a column instead of a long row, the performance will drop significantly, just like the step size curve mentioned above, it will drop by an order of magnitude. Because the main factor in improving performance is to efficiently utilize all the resources of the GPU, good use can improve performance by more than 10 times, it is still worth it. CUDA is usually used to write kernels, which is also an important factor that has led to the CUDA programming model becoming what it is today, so let's take a look at how the CUDA programming model runs parallel work.

[0046] CUDA breaks down tasks into independent blocks, each of which is run by a large number of threads together. The basic unit of execution of CUDA is the thread block, not the thread. Block is actually a quantum unit of parallelism, which means that parallelism is guaranteed to be obtained. It can be determined that all threads in the block run at the same time, so they can work together and exchange data. CUDA truly presents threads as independent, just like a single-threaded program, each thread has its own copy of all variables and its own program counter in the code. How to call this thread lies in two variables called "thread idx" and "blockidx", which are automatically filled by the hardware when the block is started to assign an index to each thread in the grid. Each thread uses its thread index and block index to calculate the data bit it is processing. After that, everything in the program is the same in all threads.

[0047] The GPU is a SIMI model, and SIMT thread control is implicit, each thread has its own state, all threads are independent, even if they are part of a vector unit, they all keep their own state, including this index variable. They can loop, branch, do conditional things, like in this example, and they are more efficient when they are doing the same task together, because things are more efficient when they are run together, and the GPU runs them in groups of 32, called warps.

[0048] In the above code, each thread loads two data points because it is looking for the distance between two points P1 and P2, which is indexed by a row that is dependent on a very important thread ID. This effectively means that I am loading data from 32 different threads at a time, with offsets indicated by some function of their thread IDs.

[0049] I'll explain why this is important later, but just know that a float2 is just a structure of two floating point numbers, eight bytes each for x and y, which is a two-dimensional coordinate. So each warp is loading 256 consecutive bytes of data as each of its 32 threads fetches a value, each indexed by a thread ID. The thread IDs are guaranteed to be consecutive, which means the data access pattern is guaranteed to be consecutive.

[0050] The SM on the GPU can manage 64 warps at a time, for a total of 2048 threads per SM. But you can see from the block diagram of the SM that it actually has four separate parts, so at any time it's actually running four warps at a time, and the others are kept in queues and switched in and out. So there are four warps, each loading 256 bytes at a time, and four times 256 bytes is 1024 bytes, which is completely coalesced adjacent data addresses loaded from the memory system at the same time, which is the perfect size to fully utilize the memory system. Because the program is reading data all over the place, it's reading from two different points arrays P1 and P2, to the computer, it looks like the program is doing random memory reads, it's reading from one array and then completely switching pages in memory to read the other. This can lose 90% of memory performance. But in fact, because of the warp execution model, because there are four running at the same time, the massively parallel GPU has exactly the right amount of data to be able to reach the peak bandwidth of the memory system even though the program is reading data from all over the place.

[0051] For the GPU, asynchronous work submission is used to keep the GPU busy. The hardware will spread the blocks as widely as possible across the SMs to maximize the requested memory bandwidth. The hardware can play a pretty good 4D Tetris, which is critical for resource packing limits, as small changes in resource utilization can have huge effects. Even using all the threads provided by the GPU, the memory system can only provide 1 / 6 of the SM request bandwidth, so the main limiting factor is memory.

[0052] We then saw that memory systems rely heavily on good access patterns, and the law of capacitance dictates that random reads of memory are more expensive than linear reads.

[0053] The GPU's Warp execution system can save overhead by reading from many threads at the same time, but it requires all threads in the Warp to process adjacent data, and all thread blocks should have at least 128 threads. This is the way Cuda is programmed, mainly to keep the GPU busy, utilize it as much as possible, and most importantly, read and write memory carefully.

[0054] The resource uniform allocation strategy is used in the initialization phase, that is, N memory resources are evenly distributed to M video streams, and each video stream corresponds to two tasks (the task types are divided into retraining tasks R and inference tasks I). For example, there are 3 memory resources (N=3), 2 video streams (M=2), and each video stream corresponds to two tasks (a total of 4 tasks), then the 3 memory resources are divided into 4 equal parts, and each task occupies 3 / 4 of the memory resources.

[0055] S20: Evaluate the resource utilization of the task, and evaluate the quality of the current allocation strategy based on the resource utilization of the task;

[0056] The resource utilization of the evaluation task and the step of evaluating the quality of the current allocation strategy based on the resource utilization of the task include: obtaining the resource utilization of at least two tasks in the current time window; calculating the global resource utilization in the current time window based on the resource utilization of at least two tasks; and evaluating the quality of the current allocation strategy based on the global resource utilization.

[0057] By evaluating the tasks, the quality of the current allocation strategy is evaluated. In order to evaluate the quality of the current allocation strategy, this embodiment sets two evaluations, namely "global resource utilization within the time window" and "global overall model accuracy within the time window". That is, N memory resources are allocated to M video streams, where each video stream corresponds to two tasks (retraining task R and inference task I). When the resource utilization of the i-th retraining task in the t-th time window is The resource utilization of the jth inference task is The global resource utilization in the tth time window is

[0058]

[0059] S30: dynamically adjust and allocate target memory resources based on the resource utilization of the task;

[0060] The step of dynamically adjusting the allocation of target memory resources based on task resource utilization includes: selecting the highest task with the highest resource utilization and the lowest task with the lowest resource utilization based on the task resource utilization; dynamically adjusting the highest task with the highest resource utilization and the lowest task with the lowest resource utilization so that the highest task with the highest resource utilization reduces the resource utilization and the lowest task with the lowest resource utilization increases the resource utilization until the highest task resource utilization is the same as the lowest task resource utilization; and setting the required memory resources corresponding to the adjusted highest task resource utilization and the lowest task resource utilization as the target memory resources.

[0061] Dynamically adjust resource allocation. In each time window, tasks with too low resource utilization or too high precision are allocated less memory resources, and tasks with too high resource utilization or too low precision are allocated more memory resources. Each GPU can be divided into G v memory resources (virtual memory resources). Based on the evaluation of the memory resource utilization and accuracy of each task, select the S / 2 retraining tasks R with the highest utilization, and then select the S / 2 inference tasks I with the lowest accuracy, and reduce 1 / G memory resources for each of the S tasks. Select the S / 2 inference I tasks with the highest accuracy, and then select the S / 2 retraining R tasks with the lowest utilization, and increase 1 / G memory resources for each of the S tasks.

[0062] This embodiment also includes: evaluating the running accuracy of the task, and evaluating the quality of the current allocation strategy based on the running accuracy of the task; and dynamically adjusting the allocation as the target memory resource based on the running accuracy of the task.

[0063] By evaluating the tasks, the quality of the current allocation strategy is evaluated. In order to evaluate the quality of the current allocation strategy, this embodiment sets two evaluations, namely "global resource utilization within the time window" and "global overall model accuracy within the time window". That is, N memory resources are allocated to M video streams, where each video stream corresponds to two tasks (retraining task R and inference task I). When the model accuracy of the i-th retraining task in the t-th time window is The model accuracy for the jth inference task is The global model accuracy of the tth time window is

[0064]

[0065] The step of dynamically adjusting the allocation as the target memory resource based on the task's running accuracy includes: selecting a high task with the highest running accuracy and a lowest task with the lowest running accuracy based on the task's running accuracy; dynamically adjusting the highest task with the highest running accuracy and the lowest task with the lowest running accuracy so that the highest task with the highest running accuracy reduces the running accuracy and the lowest task with the lowest running accuracy increases the running accuracy until the running accuracy of the highest task is the same as the running accuracy of the lowest task; and setting the required memory resources corresponding to the adjusted running accuracy of the highest task and the running accuracy of the lowest task as the target memory resources.

[0066] The step of dynamically adjusting the configuration of tasks based on the target memory resources includes: dynamically adjusting, based on the target memory resources, in each time window, tasks with too high precision to lower the task configuration level and tasks with too low precision to increase the task configuration level.

[0067] The step of dynamically adjusting, based on the target memory resources, the tasks with too high precision in each time window to lower the task configuration gear and the tasks with too low precision to increase the task configuration gear includes: when the task is a retraining task, adjusting the gear of the retraining task by dynamically adjusting the "number of iterations": when the task is an inference task, adjusting the sampling rate and resolution of the video frame to adjust the gear of the task.

[0068] S40: Dynamically adjust the configuration of tasks based on target memory resources.

[0069] In each time window, tasks with too high precision have their task configuration gear lowered (the higher the task configuration gear, the higher the precision, but the more resources consumed), and tasks with too low precision have their task configuration gear increased (the lower the task configuration gear, the fewer resources consumed, but the precision will be reduced).

[0070] For the retraining task R, you can adjust the gear of the retraining task by dynamically adjusting the "number of iterations". If the maximum gear of this type of task is The minimum gear is If there are L gears in total, then each gear is Each time, find the retraining task R with the lowest accuracy, and then upgrade the configuration of the task, as follows

[0071]

[0072] The min(·) function is used to ensure that the gear does not exceed the maximum gear. At the same time, find the retraining task R with the highest accuracy and downgrade the configuration of this task as follows:

[0073]

[0074] The max(·) function is used to ensure that the gear position does not fall below the minimum gear.

[0075] Similarly, for inference task I, the task gear can be adjusted by adjusting the sampling rate and resolution of the video frame. If the maximum gear of this type of task is The minimum gear is If there are L gears in total, then each gear is Each time we find the inference task with the lowest accuracy, we file the configuration of the task as follows

[0076]

[0077] The min(·) function is used to ensure that the gear does not exceed the maximum gear. At the same time, find the inference task with the highest accuracy and downgrade the configuration of the task as follows:

[0078]

[0079] The max(·) function is used to ensure that the gear position does not fall below the minimum gear.

[0080] Repeat iterative steps S10-S40 to implement a dynamic task adjustment strategy.

[0081] The present invention provides a task scheduling method for continuous learning from edge monitoring video streams, first by allocating initial memory resources to tasks according to a uniform allocation strategy; evaluating the resource utilization of tasks, and evaluating the quality of the current allocation strategy based on the resource utilization of tasks; dynamically adjusting the allocation to target memory resources based on the resource utilization of tasks; and dynamically adjusting the configuration of tasks based on target memory resources. In order to maximize the global resource utilization and global model accuracy, in each time window, tasks with too low resource utilization are allocated fewer memory resources, tasks with too high accuracy are reduced in task configuration gears (the higher the task configuration gear, the higher the accuracy, but the more resources consumed), tasks with too high resource utilization are allocated more memory resources, and tasks with too low accuracy are increased in task configuration gears (the lower the task configuration gear, the less resource consumption, but the accuracy will be reduced). After the improvement, the global resource utilization and global model accuracy can be improved.

[0082] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0083] In one embodiment, a task scheduling device for continuously learning from edge monitoring video streams is provided, and the task scheduling device for continuously learning from edge monitoring video streams corresponds one-to-one to the task scheduling method for continuously learning from edge monitoring video streams in the above embodiment. Figure 3 As shown, the task scheduling device for continuous learning from edge monitoring video streams includes an initial allocation module 101, an evaluation module 102, a dynamic resource adjustment module 103, and a dynamic configuration adjustment module 104. The functional modules are described in detail as follows:

[0084] An initial allocation module 101 is used to allocate initial memory resources to tasks according to a uniform allocation strategy;

[0085] An evaluation module 102 is used to evaluate the resource utilization of the task, and evaluate the quality of the current allocation strategy based on the resource utilization of the task;

[0086] A dynamic resource adjustment module 103 is used to dynamically adjust the resource allocation as a target memory resource based on the resource utilization of the task;

[0087] The dynamic configuration adjustment module 104 is used to dynamically adjust the configuration of the task based on the target memory resources.

[0088] In one embodiment, the allocation module 101 is specifically configured to:

[0089] The acquisition submodule is used to obtain the resource utilization of at least two tasks in the current time window;

[0090] A calculation submodule, used to calculate and obtain the global resource utilization in the current time window according to the resource utilization of at least two tasks;

[0091] The resource utilization evaluation submodule is used to evaluate the quality of the current allocation strategy based on the global resource utilization.

[0092] In one embodiment, the allocation module 101 is further configured to:

[0093] A resource utilization selection submodule is used to select the highest task with the highest resource utilization and the lowest task with the lowest resource utilization based on the resource utilization of the task;

[0094] The resource utilization dynamic adjustment submodule is used to dynamically adjust the highest task with the highest resource utilization and the lowest task with the lowest resource utilization, so that the highest task with the highest resource utilization reduces resource utilization, and the lowest task with the lowest resource utilization increases resource utilization, until the highest task resource utilization is the same as the lowest task resource utilization; the memory resources required corresponding to the adjusted highest task resource utilization and the lowest task resource utilization are set as the target memory resources.

[0095] In one embodiment, the device is further configured to:

[0096] The accuracy evaluation module is used to evaluate the running accuracy of the task and evaluate the quality of the current allocation strategy based on the running accuracy of the task;

[0097] The dynamic precision adjustment module is used to dynamically adjust the target memory resources allocated based on the running precision of the task.

[0098] In one embodiment, the dynamic adjustment accuracy module is further used to:

[0099] A precision selection submodule is used to select a high task with the highest running precision and a lowest task with the lowest running precision based on the running precision of the task;

[0100] The precision dynamic adjustment submodule is used to dynamically adjust the highest task with the highest operating precision and the lowest task with the lowest operating precision, so that the highest task with the highest operating precision reduces its operating precision and the lowest task with the lowest operating precision improves its operating precision until the highest task operating precision is the same as the lowest task operating precision; the memory resources required corresponding to the adjusted highest task operating precision and the lowest task operating precision are set as the target memory resources.

[0101] In one embodiment, the dynamic adjustment configuration module 104 is specifically used to:

[0102] The precision dynamic adjustment submodule is used to dynamically adjust the tasks with too high precision in each time window to lower the task configuration gear and to increase the task configuration gear for tasks with too low precision based on the target memory resources.

[0103] In one embodiment, the precision dynamic adjustment submodule is specifically used for:

[0104] The retraining task unit is used to adjust the gear of the retraining task by dynamically adjusting the "number of iterations" when the task is a retraining task:

[0105] The inference task unit is used to adjust the sampling rate and resolution of the video frame to adjust the gear of the task when the task is an inference task.

[0106] The present invention provides a task scheduling device for continuous learning from edge monitoring video streams, first by allocating initial memory resources to tasks according to a uniform allocation strategy; evaluating the resource utilization of tasks, and evaluating the quality of the current allocation strategy based on the resource utilization of tasks; dynamically adjusting the allocation to target memory resources based on the resource utilization of tasks; and dynamically adjusting the configuration of tasks based on target memory resources. In order to maximize the global resource utilization and global model accuracy, in each time window, tasks with too low resource utilization are allocated fewer memory resources, tasks with too high accuracy are reduced in task configuration gear (the higher the task configuration gear, the higher the accuracy, but the more resources consumed), tasks with too high resource utilization are allocated more memory resources, and tasks with too low accuracy are increased in task configuration gear (the lower the task configuration gear, the less resource consumption, but the accuracy will be reduced). After the improvement, the global resource utilization and global model accuracy can be improved.

[0107] For the specific limitations of the task scheduling device for continuous learning from edge monitoring video streams, please refer to the limitations of the task scheduling method for continuous learning from edge monitoring video streams above, which will not be repeated here. Each module in the above-mentioned task scheduling device for continuous learning from edge monitoring video streams can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0108] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it realizes the functions or steps of the server side of a task scheduling method for continuous learning from an edge monitoring video stream.

[0109] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps of the client side of a task scheduling method for continuous learning from an edge monitoring video stream

[0110] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0111] Allocate initial memory resources to tasks according to the uniform allocation strategy;

[0112] Evaluate the resource utilization of tasks and evaluate the quality of the current allocation strategy based on the resource utilization of tasks;

[0113] Dynamically adjust the memory resources allocated as the target based on the resource utilization of the task;

[0114] Dynamically adjust the configuration of tasks based on target memory resources.

[0115] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0116] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0117] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0118] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A task scheduling method for continuous learning from edge monitoring video streams, characterized in that: include: Allocate initial memory resources to tasks according to the uniform allocation strategy; Evaluate the resource utilization of tasks and evaluate the quality of the current allocation strategy based on the resource utilization of tasks; Dynamically adjust the memory resources allocated as the target based on the resource utilization of the task; Dynamically adjust the configuration of tasks based on target memory resources.

2. The task scheduling method for continuous learning from edge monitoring video streams as claimed in claim 1, characterized in that: The step of evaluating the resource utilization of the task and evaluating the quality of the current allocation strategy based on the resource utilization of the task includes: In the current time window, obtain the resource utilization of at least two tasks; The global resource utilization in the current time window is obtained based on the resource utilization of at least two tasks; Evaluate the quality of the current allocation strategy based on global resource utilization.

3. The task scheduling method for continuous learning from edge monitoring video streams as claimed in claim 2, characterized in that: The step of dynamically adjusting the allocation of target memory resources based on the resource utilization of the task comprises: Based on the resource utilization of the tasks, the highest task with the highest resource utilization and the lowest task with the lowest resource utilization are selected; Dynamically adjust the highest task with the highest resource utilization and the lowest task with the lowest resource utilization, so that the highest task with the highest resource utilization reduces resource utilization and the lowest task with the lowest resource utilization increases resource utilization until the resource utilization of the highest task is the same as that of the lowest task; The memory resources required by the adjusted maximum task resource utilization and the minimum task resource utilization are set as target memory resources.

4. The task scheduling method for continuous learning from edge monitoring video streams as claimed in claim 1, characterized in that: After the step of allocating initial memory resources to tasks according to the uniform allocation strategy, the method further includes: Evaluate the running accuracy of the task, and evaluate the quality of the current allocation strategy based on the running accuracy of the task; Dynamically adjust the memory resources allocated as the target based on the execution accuracy of the task.

5. The task scheduling method for continuous learning from edge monitoring video streams as claimed in claim 4, characterized in that: The step of dynamically adjusting the allocation of target memory resources based on the task's running accuracy comprises: Based on the running accuracy of the tasks, a high task with the highest running accuracy and a lowest task with the lowest running accuracy are selected; Dynamically adjust the highest task with the highest operating accuracy and the lowest task with the lowest operating accuracy, so that the highest task with the highest operating accuracy reduces its operating accuracy and the lowest task with the lowest operating accuracy improves its operating accuracy, until the highest task operating accuracy is the same as the lowest task operating accuracy; The memory resources required by the adjusted highest task running accuracy and the lowest task running accuracy are set as target memory resources.

6. The task scheduling method for continuous learning from edge monitoring video streams as claimed in claim 5, characterized in that: The step of dynamically adjusting the configuration of the task based on the target memory resource comprises: Based on the target memory resources, dynamically adjust the task configuration level of tasks with too high precision in each time window to lower the task configuration level, and increase the task configuration level of tasks with too low precision.

7. The task scheduling method for continuous learning from edge monitoring video streams as claimed in claim 6, characterized in that: The step of dynamically adjusting, based on the target memory resources, the tasks with too high precision in each time window to lower the task configuration gear and the tasks with too low precision to increase the task configuration gear comprises: When the task is a retraining task, the gear of the retraining task is adjusted by dynamically adjusting the "number of iterations": When the task is an inference task, the sampling rate and resolution of the video frame are adjusted to adjust the gear of the task.

8. A task scheduling device for continuous learning from edge monitoring video streams, characterized in that: include: The initial allocation module is used to allocate initial memory resources to tasks according to the uniform allocation strategy; The evaluation module is used to evaluate the resource utilization of the task and evaluate the quality of the current allocation strategy based on the resource utilization of the task; A dynamic resource adjustment module is used to dynamically adjust the allocation of target memory resources based on the resource utilization of the task; The dynamic configuration adjustment module is used to dynamically adjust the configuration of tasks based on target memory resources.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the task scheduling method for continuous learning from the edge monitoring video stream are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the task scheduling method for continuous learning from edge monitoring video streams as described in any one of claims 1 to 7 are implemented.

Images