Dynamic resource regulation system based on computing power reservation and container mapping

Abstract

This invention discloses a dynamic resource control system based on computing power reservation and container mapping, belonging to the field of resource control technology. It is used to solve the problems of response delay of transcoding tasks during peak periods and idle and wasted computing power resources during off-peak periods. By retrieving historical load data of the computing power cluster, peak and off-peak computing power periods are identified. Based on task attributes, video frame rate and category, batch transcoding tasks are allocated to different computing power periods for execution. During the computing power reservation period, resource mapping and execution monitoring are performed on running tasks, and task completion rate and resource usage data are collected in real time. Dynamic reallocation of task runtime periods is achieved through computing power adaptation scoring. Throughout the task execution process, the matching relationship between tasks and computing power periods is continuously corrected, so that the resource utilization status and task processing efficiency are maintained in a more stable balance range, thereby maintaining the balance of the overall throughput performance and computing power distribution of the cluster in a multi-type task concurrent environment.

Description

Technical Field

[0001] This invention relates to the field of resource regulation technology, and more specifically, to a dynamic resource regulation system based on computing power reservation and container mapping. Background Technology

[0002] With the continuous growth of various data processing needs, especially the widespread application of large-scale media processing, format conversion and batch processing computing tasks, the volume of related data processing business is showing a continuous upward trend. Existing systems usually adopt a static resource scheduling strategy, which allocates tasks to computing nodes in the computing power cluster according to fixed rules.

[0003] The existing technology has the following shortcomings:

[0004] Currently, existing technologies only use static task allocation strategies for transcoding, lacking a dynamic scheduling mechanism based on computing load and task attributes. This makes it impossible to achieve time-dimensional optimization of computing resources during peak and off-peak periods and adaptive task control, resulting in delayed transcoding task response during peak periods, idle and wasted computing resources during off-peak periods, and increased system energy consumption. Therefore, a dynamic resource control system based on computing power reservation and container mapping is proposed.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a dynamic resource control system based on computing power reservation and container mapping. This system addresses the problems mentioned in the background art by employing historical load analysis of computing power clusters, peak and off-peak time period division, task attribute and video frame rate identification, computing power time period matching, resource mapping and execution monitoring, dynamic computing power adaptation scoring, and task time period adjustment.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a dynamic resource control system based on computing power reservation and container mapping, comprising a time period division module, a time period matching module, a computing power scheduling module, and a computing power optimization module, the functions of each module being as follows:

[0008] Before a user submits a batch transcoding task, the time period segmentation module retrieves historical load data from the computing power cluster to analyze the computing power reserved time period, identifies peak computing power periods and off-peak computing power periods based on the computing power reserved time period, and passes them to the matching time period module.

[0009] The matching time period module performs video attribute parsing and marks the transcoding category for batch transcoding tasks, accesses the category mapping database to match the target computing power time period corresponding to the transcoding category, detects the video frame rate of each transcoding task, and allocates different computing power time periods to the transcoding tasks in combination with the target computing power time period and passes them to the computing power scheduling module.

[0010] When the computing power reservation period is in effect, the computing power scheduling module performs resource mapping processing on transcoding tasks, detects the task throughput of different computing power periods and calculates the task completion rate, counts the number of transcoding tasks corresponding to different computing power periods and evaluates the resource occupancy characteristics, and transmits the task completion rate and resource occupancy characteristics to the computing power optimization module.

[0011] After receiving resource usage characteristics and task completion rates, the computing power optimization module generates a computing power adaptation score. Based on the computing power adaptation score, it filters and marks computing power time periods, selects transcoding tasks to be processed within the marked computing power time periods, and determines whether to change the computing power time period.

[0012] In a preferred embodiment, in the time period segmentation module, before the user submits the batch transcoding task, a statistical period is preset and divided into multiple statistical intervals.

[0013] Within a preset statistical period, historical load data of the computing power cluster is retrieved through the computing power monitoring interface. The historical load data includes CPU utilization and task queue length.

[0014] After standardizing the CPU utilization and task queue length, we obtain the CPU utilization coefficient and the task queue coefficient.

[0015] If the CPU utilization coefficient is greater than the preset CPU utilization threshold within the statistical interval, the statistical interval is marked.

[0016] Conversely, the statistical interval is classified as an off-peak period.

[0017] In a preferred embodiment, in the time period segmentation module, the average value of the task queue coefficients within each statistical interval is taken to obtain the task queue mean.

[0018] If the task queue coefficient within the marked statistical interval is greater than the task queue mean, then the category of the marked statistical interval is determined to be a peak period.

[0019] Conversely, the statistical interval is classified as an off-peak period.

[0020] After obtaining the category results for each statistical interval, adjacent statistical intervals with consistent category results are merged to obtain peak computing power periods or off-peak computing power periods.

[0021] Within a preset statistical period, the off-peak computing power period and peak computing power period of the day are vertically compared and overlapped on the time axis to count the frequency of occurrence of category results for the same period on different dates;

[0022] Adjust peak computing power periods or off-peak computing power periods based on the frequency of occurrence.

[0023] In a preferred embodiment, in the matching time period module, video attributes of the batch transcoding tasks are parsed through the video parsing interface to obtain the video file attribute information of each transcoding task, including the target transcoding specification and encoding format.

[0024] Input the target transcoding specification and encoding format into the preset format mapping table, and output the transcoding category of the transcoding task;

[0025] The access category mapping database matches the target computing power time period corresponding to the transcoding category. The target computing power time period includes the priority scheduling time period and the delayed execution time period.

[0026] In a preferred embodiment, the matching time period module presets basic weights for the priority scheduling time period and the delayed execution time period respectively;

[0027] The video frame rate of each transcoding task is obtained through the video parsing interface. After the video frame rate of the transcoding task is standardized, it is multiplied by the basic weight corresponding to the transcoding task to obtain the time period adaptation value.

[0028] If the time period adaptation value is greater than the preset time period adaptation threshold, the transcoding task will be assigned to the peak computing power period.

[0029] Conversely, transcoding tasks will be allocated to off-peak computing time periods.

[0030] In a preferred embodiment, in the computing power scheduling module, when the computing power reservation period is in effect, the transcoding task and its corresponding computing power period output by the matching period module are received, and resource mapping processing is performed on each transcoding task according to the available resource status of each node in the computing power cluster.

[0031] After completing the resource mapping process, the number of video frames completed by each transcoding task in each computing power period is counted and used as the task throughput of each transcoding task in different computing power periods.

[0032] The total frame processing volume of all transcoding tasks during the computing power period is obtained by summing up the frame processing volume of all transcoding tasks during the computing power period.

[0033] The ratio of task throughput to total frame processing volume is used as the task completion rate.

[0034] In a preferred embodiment, in the computing power scheduling module, all transcoding tasks currently in execution are extracted from the task scheduling record, and the transcoding tasks are grouped and counted according to the computing power time period identifier to which the transcoding tasks belong, so as to obtain the number of transcoding tasks corresponding to each computing power time period.

[0035] The maximum number of tasks to run is retrieved from the resource configuration database. The maximum number of tasks to run is the maximum number of tasks that the cluster can run simultaneously under the current configuration conditions.

[0036] The ratio of the number of transcoding tasks to the maximum number of tasks can be used as a resource consumption characteristic.

[0037] In a preferred embodiment, in the computing power optimization module, after receiving the resource occupancy characteristics and task completion rate, they are standardized to obtain the resource occupancy factor and the task rate factor.

[0038] The resource occupancy factor and task rate factor are calculated comprehensively using a weighted summation algorithm to obtain a computing power matching score.

[0039] If the computing power adaptation score is greater than the preset adaptation score threshold, the computing power period is marked.

[0040] Conversely, if the computing power matching score is less than or equal to the preset matching score threshold, the computing power period will not be marked.

[0041] In a preferred embodiment, in the computing power optimization module, for a marked computing power period, transcoding tasks in the execution state are retrieved from the task scheduling record to obtain transcoding tasks to be processed, and the task throughput of transcoding tasks to be processed in the marked computing power period is detected.

[0042] The average throughput of transcoding tasks across all computing power periods is used as the baseline throughput.

[0043] If the throughput of the transcoding task to be processed is less than the baseline throughput, the transcoding task to be processed will be marked as a portable task, and a computing power time period change instruction will be generated.

[0044] The technical effects and advantages of this invention are as follows:

[0045] This invention identifies peak and off-peak computing power periods by retrieving historical load data from the computing power cluster. Based on task attributes, video frame rate, and category, it allocates batch transcoding tasks to different computing power periods for execution. During the period when computing power reservation is in effect, it performs resource mapping and execution monitoring on running tasks, collects task completion rate and resource usage data in real time, and dynamically evaluates computing power suitability based on the calculation results. When uneven computing power utilization or decreased task processing efficiency is detected, the computing power period where the task is located is adjusted. This achieves a matching of computing power supply and demand for video transcoding tasks in the time dimension, ensuring the response speed of transcoding tasks during peak periods and making full use of idle computing power during off-peak periods, significantly improving computing power utilization and system throughput efficiency, and reducing energy waste. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the implementation of the dynamic resource control system based on computing power reservation and container mapping of the present invention.

[0047] Figure 2 This is a module framework diagram of the dynamic resource regulation system based on computing power reservation and container mapping of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] This invention identifies peak and off-peak computing power periods by retrieving historical load data from the computing power cluster. Based on task attributes, video frame rate, and category, it allocates batch transcoding tasks to different computing power periods for execution. During the period when computing power reservation is in effect, it performs resource mapping and execution monitoring on running tasks, collects task completion rate and resource usage data in real time, and dynamically evaluates computing power suitability based on the calculation results. When uneven computing power utilization or decreased task processing efficiency is detected, the computing power period where the task is located is adjusted. This achieves a time-dimensional matching of computing power supply and demand for video transcoding tasks, ensuring the response speed of transcoding tasks during peak periods and making full use of idle computing power during off-peak periods, significantly improving computing power utilization and system throughput efficiency.

[0050] Example 1, such as Figures 1 to 2 As shown, the dynamic resource control system based on computing power reservation and container mapping includes a time-segmentation module, a time-segmentation matching module, a computing power scheduling module, and a computing power optimization module. These modules are electrically connected, and their functions are as follows:

[0051] Before a user submits a batch transcoding task, the time period segmentation module retrieves historical load data from the computing power cluster to analyze the computing power reserved time period, identifies peak computing power periods and off-peak computing power periods based on the computing power reserved time period, and passes them to the matching time period module.

[0052] The matching time period module performs video attribute parsing and marks the transcoding category for batch transcoding tasks, accesses the category mapping database to match the target computing power time period corresponding to the transcoding category, detects the video frame rate of each transcoding task, and allocates different computing power time periods to the transcoding tasks in combination with the target computing power time period and passes them to the computing power scheduling module.

[0053] When the computing power reservation period is in effect, the computing power scheduling module performs resource mapping processing on transcoding tasks, detects the task throughput of different computing power periods and calculates the task completion rate, counts the number of transcoding tasks corresponding to different computing power periods and evaluates the resource occupancy characteristics, and transmits the task completion rate and resource occupancy characteristics to the computing power optimization module.

[0054] After receiving resource usage characteristics and task completion rates, the computing power optimization module generates a computing power adaptation score. Based on the computing power adaptation score, it filters and marks computing power time periods, selects transcoding tasks to be processed within the marked computing power time periods, and determines whether to change the computing power time period.

[0055] The specific implementation is as follows:

[0056] In the time-segmentation module, the load status of computing resources is analyzed historically and predicted periodically before users submit batch transcoding tasks, forming computing time periods that can be used for transcoding task allocation.

[0057] The statistical period is preset and divided into multiple statistical intervals. Within the preset statistical period, the historical load data of the computing power cluster is retrieved through the computing power monitoring interface. The historical load data includes CPU utilization and task queue length.

[0058] CPU utilization reflects the real-time computing power usage of each node in the computing power cluster, while task queue length reflects the degree of task backlog and resource scheduling pressure of the computing power cluster.

[0059] After standardizing the CPU utilization and task queue length, we obtain the CPU utilization coefficient and the task queue coefficient.

[0060] The CPU utilization threshold is compared with the CPU utilization coefficient. If the CPU utilization coefficient is greater than the preset CPU utilization threshold within the statistical interval, the statistical interval is marked; otherwise, the statistical interval is classified as an off-peak period.

[0061] The average value of the task queue is obtained by averaging the task queue coefficients within each statistical interval.

[0062] If the task queue coefficient within the marked statistical interval is greater than the task queue mean, then the marked statistical interval is classified as a peak period; otherwise, the statistical interval is classified as an off-peak period.

[0063] After obtaining the category results of each statistical interval, adjacent peak periods are merged. When multiple adjacent statistical intervals are all determined to be peak periods, they are merged to form a continuous peak computing power period. Similarly, adjacent off-peak periods are merged to obtain off-peak computing power periods.

[0064] For peak or off-peak periods that are not merged, the statistical interval category results are merged into adjacent intervals with consistent categories to ensure the continuity of peak computing power periods and off-peak computing power periods on the time axis.

[0065] Peak computing power periods refer to time intervals with high computing power resource utilization, used to execute transcoding tasks with high computational intensity or high real-time requirements, ensuring the processing speed of transcoding tasks and the quality of video output; off-peak computing power periods refer to time intervals with low computing power resource utilization, used to execute transcoding tasks with low computational intensity or no real-time requirements, making full use of idle computing power resources and improving the overall resource utilization rate.

[0066] Within a preset statistical period, the off-peak computing power period and peak computing power period of a day are vertically compared and overlapped on the time axis to count the frequency of occurrence in multi-day data.

[0067] In vertical analysis, if the computing power period category results are consistent across multiple dates and the same time period, the category result is directly retained as the final category result. If there are differences in the category results within the same time period, the frequency of occurrence of different category results is statistically compared, and the category result with the higher frequency is determined as the final computing power period for that time period.

[0068] It should be explained that the preset statistical period is used to limit the time range of computing load analysis, which can be set to multiple consecutive days; the computing load monitoring interface is used to retrieve running load data from the computing cluster; the computing cluster refers to a distributed computing resource collection composed of multiple computing nodes, with GPU computing units as the main computing resources in its computing nodes. Each node shares resources and performs collaborative computing through a task scheduling system, which is the basic execution carrier for computing power reservation and task allocation; the standardization processing methods include, but are not limited to, standard linear transformation based on interval scaling, Z-Score standardization method based on statistics, or normalization method based on nonlinear mapping function. The application methods of standardization processing will not be elaborated here; the preset CPU utilization threshold can be set according to the node performance distribution, historical peak utilization rate, and task response latency of the computing cluster.

[0069] Through the above process, the trend of computing load change can be identified before the transcoding task is submitted, and the peak and off-peak resources can be predicted and reserved. This enables the pre-emptive scheduling of computing power and the controllability of the time dimension, avoiding sudden computing power shortages during peak periods or idle resources during off-peak periods, thereby improving the overall computing power utilization and task scheduling efficiency.

[0070] In the matching time period module, after identifying peak computing power periods and off-peak computing power periods, the video attributes of the batch transcoding tasks submitted by users are parsed and the transcoding category is marked.

[0071] The video attribute is parsed through the video parsing interface to obtain the video file attribute information of each transcoding task, including the target transcoding specification and encoding format.

[0072] Among them, the target transcoding specification refers to the parameters output by the transcoding task after transcoding is completed; it is used to limit the processing intensity and output standard of the transcoding task; the encoding format refers to the compression encoding method used by the video file, and different encoding formats have different compression complexity and hardware decoding compatibility.

[0073] Input the target transcoding specification and encoding format into the preset format mapping table, and output the transcoding category of the transcoding task. The transcoding categories include archiving transcoding and distribution optimization transcoding, etc.

[0074] It should be explained that the preset format mapping table is used to establish the correspondence between transcoding specifications and encoding formats. It is generated by statistical results of historical transcoding tasks. The table records the transcoding categories corresponding to different combinations of transcoding specifications and encoding formats. For example, when the target transcoding specification is 1080P and the encoding format is H.264, the output transcoding category is distribution optimization transcoding.

[0075] Furthermore, for training and inference-related tasks, a new task attribute parsing dimension is added to extract features such as model parameter scale, batch inference frame count, and training iteration requirements. These features are then incorporated into a format mapping table and marked as training and inference collaborative tasks. High-priority model training tasks, which require continuous and stable computing power support, are by default bound to dedicated computing power reserved quotas for priority scheduling periods. Low-latency inference tasks ensure resource supply during peak periods by increasing basic weights. Non-urgent training tasks are adapted to off-peak periods for computing power filling, realizing collaborative scheduling of computing power resources between transcoding and training / inference tasks. This will not be elaborated further here.

[0076] Access category mapping database matches the target computing power time period corresponding to the transcoding category. The target computing power time period includes priority scheduling time period and delayed execution time period. The target computing power time period refers to the computing power execution time interval set according to the transcoding category, which is used to schedule transcoding tasks of different transcoding categories in the time dimension.

[0077] For example, priority scheduling periods correspond to real-time transcoding categories with high real-time requirements or high computational demands, used to ensure the processing performance of tasks such as high-definition video output, while delayed execution periods correspond to archive transcoding categories with lower computational intensity or that can be delayed.

[0078] Preset base weights for priority scheduling periods and delayed execution periods, with the base weight for priority scheduling periods being higher than that for delayed execution periods;

[0079] The video frame rate of each transcoding task is obtained through the video parsing interface. The higher the video frame rate, the greater the computing power and memory consumption of the transcoding task during the transcoding process.

[0080] The time period adaptation value is obtained by multiplying the video frame rate of the transcoding task by the basic weight corresponding to the transcoding task after standardization.

[0081] The time period adaptation value is compared with the preset time period adaptation threshold to allocate different computing power time periods for transcoding tasks:

[0082] If the time period adaptation value is greater than the preset time period adaptation threshold, the transcoding task will be assigned to the peak computing power period.

[0083] Conversely, transcoding tasks will be allocated to off-peak computing time periods.

[0084] When the time period adaptation value is greater than the preset time period adaptation threshold, it indicates that the transcoding task requires a high computing power intensity. It should be allocated to the peak computing power period for execution to shorten the transcoding time and maintain stable output quality.

[0085] It should be explained that the video parsing interface is a functional interface used to perform structured feature extraction on the transcoding task to be processed; the category mapping database is a data module that stores the correspondence between different transcoding categories and target computing power time periods; the preset basic weight is the initial weight set for different target computing power time periods, which can be set according to the distribution pattern of transcoding tasks; the preset time period adaptation threshold can be set according to historical running data, for example, based on the statistical results of the average video frame rate distribution of transcoding tasks in a recent period.

[0086] This module identifies transcoding categories by parsing the transcoding specifications and encoding formats of transcoding tasks. It then determines the target computing time period through a category mapping database. Combining this with the video frame rate, it calculates the time period adaptation value and compares it with the adaptation threshold. High frame rate, high load tasks are assigned to peak computing time periods for execution, while low load, delayable tasks are assigned to off-peak computing time periods for processing. During peak periods, this effectively reduces task queuing and latency risks, ensuring real-time transcoding performance. During off-peak periods, it improves system resource utilization and reduces computing power waste.

[0087] In the computing power scheduling module, when the computing power reservation period is in effect, it receives the transcoding tasks and their corresponding computing power periods output by the matching period module, and performs resource mapping processing on each transcoding task based on the available resource status of each node in the computing power cluster.

[0088] Specifically, each transcoding task is formally assigned to a specific computing time slot and transcoded. This process is based on a containerized runtime environment. According to the resource requirements of the transcoding task, the real-time available resources of each node are matched, and the node that meets the requirements of the transcoding task and has the optimal load is selected. The transcoding task container is instantiated on the corresponding node, the transcoding image and dependent environment required by the transcoding task are loaded, and the initialization and resource binding of the transcoding task runtime context are completed, thereby realizing the resource mapping between the transcoding task and the node.

[0089] It should be noted that a containerized runtime environment refers to a virtualized execution environment used to encapsulate, isolate, and manage transcoding task execution resources. Through container technology, it achieves lightweight resource isolation and rapid deployment at the application layer, and provides a unified task execution interface and resource scheduling interface. In this environment, computing resources such as CPU, memory, I / O, and GPU are allocated to different transcoding task containers as needed, ensuring that tasks do not interfere with each other when running on the same computing node. Container instantiation refers to the process by which the computing power scheduling module maps transcoding tasks to specific computing nodes and starts running instances after determining the nodes.

[0090] After completing the resource mapping process, the computing power scheduling module monitors the execution status of transcoding tasks in each computing power period to obtain processing progress data of transcoding tasks in different computing power periods.

[0091] In the specific process, task execution details are extracted from the task scheduling records of each node, including transcoding task identifier, start and end time, number of processed frames and computing power period identifier. Then, the transcoding tasks are classified and organized according to computing power period, and the number of video frames completed by each transcoding task in each computing power period is counted, which is used as the task throughput of each transcoding task in different computing power periods.

[0092] The total frame processing volume of all transcoding tasks during the computing power period is obtained by summing up the frame processing volume of all transcoding tasks during the computing power period.

[0093] The ratio of task throughput to total frame processing volume is used as the task completion rate, reflecting the progress and completion rate of each transcoding task within the computing power period.

[0094] The task throughput is statistically analyzed based on the number of video frames generated by GPU parallel processing, and the calculated task completion rate is used to reflect the execution efficiency under GPU acceleration conditions.

[0095] It should be noted that the task scheduling record is a set of data records generated and continuously updated by the computing power scheduling module during task execution. It is used to reflect the allocation, execution and completion status of each transcoding task in the computing power cluster.

[0096] Subsequently, the computing power scheduling module statistically analyzes the task distribution of each computing power time period and assesses the resource occupancy characteristics accordingly.

[0097] Specifically, all transcoding tasks currently in execution are extracted from the task scheduling records. The transcoding tasks are grouped and statistically analyzed according to the computing power time period identifier to which they belong, so as to obtain the number of transcoding tasks corresponding to each computing power time period, which reflects the task concurrency of the computing power time period.

[0098] The maximum number of tasks to run is retrieved from the resource configuration database. The maximum number of tasks to run is the maximum number of tasks that the cluster can run simultaneously under the current configuration conditions. The maximum number of tasks to run can be determined based on the parallel processing capability of the GPU, the video memory capacity, or the number of hardware coding sessions.

[0099] The ratio of the number of transcoding tasks to the maximum task execution volume is used as a resource usage characteristic to reflect the overall resource load caused by the number of transcoding tasks.

[0100] The task completion rate and resource usage characteristics are passed to the computing power optimization module.

[0101] In the computing power optimization module, after receiving the resource occupancy characteristics and task completion rate, they are standardized to obtain the resource occupancy factor and task rate factor.

[0102] The resource occupancy factor and task rate factor are comprehensively calculated using a weighted summation algorithm to obtain the computing power suitability score, as shown in the following expression:

[0103] ;

[0104] in, Scoring based on computing power adaptation. For task rate factor, For resource occupancy factor, and These are the preset weighting coefficients.

[0105] The computing power adaptation score reflects the execution efficiency and resource matching degree of computing power during a given time period under the current resource configuration. The higher the score, the better the computing power utilization during that time period.

[0106] The computing power matching score is compared with the preset matching score threshold:

[0107] If the computing power adaptation score is greater than the preset adaptation score threshold, the computing power period is marked.

[0108] Conversely, if the computing power matching score is less than or equal to the preset matching score threshold, the computing power period will not be marked.

[0109] It should be noted that the weighting coefficient is a parameter used to balance the influence of the task rate factor and resource consumption factor on the computing power adaptation score, and it must satisfy the constraints. By retrieving historical task execution logs and computing power monitoring data, the sensitivity coefficients of task completion rate to overall transcoding latency and resource usage changes to average load of computing power clusters under different computing power configurations are calculated. Based on the proportional relationship between the two types of sensitivity coefficients, a normalization method is used to determine the weighting coefficients. The adaptation score threshold is a benchmark parameter used to determine whether the computing power is in a low adaptation state during a certain period. In the setting process, the distribution range of computing power adaptation scores during a certain period is calculated based on long-term running data. The mean and standard deviation of the computing power adaptation scores are calculated, and the difference between the mean and standard deviation is used as the adaptation score threshold.

[0110] This module implements a unified scheduling and management mechanism for multiple types of batch processing tasks. It can reflect the computing power usage status of the cluster in a timely manner according to load changes, and provide accurate and quantifiable operating characteristics for subsequent adaptation scoring calculations, so that the system can maintain stable scheduling capabilities and resource utilization efficiency under both high and low load conditions.

[0111] For the marked computing power period, the transcoding tasks in the execution state are retrieved from the task scheduling record to obtain the transcoding tasks to be processed, and the task throughput of the transcoding tasks to be processed in the marked computing power period is detected.

[0112] The average throughput of transcoding tasks across all computing power periods is used as the baseline throughput.

[0113] If the throughput of the transcoding task to be processed is less than the baseline throughput, the transcoding task to be processed is marked as a portable task, and a computing power time slot change instruction is generated to allocate the transcoding task to other computing power time slots for execution.

[0114] By using computing power adaptation scoring, dynamic reallocation of task runtime segments is achieved. Throughout the entire task execution process, the matching relationship between tasks and computing power time periods is continuously corrected, so that resource utilization and task processing efficiency are maintained in a more stable balance range. This ensures the overall throughput performance and computing power distribution of the cluster remain balanced in a multi-type task concurrent environment.

[0115] This invention is applicable to various computing power environments. Through a unified computing power reservation and dynamic adjustment mechanism, it achieves flexible adaptation to heterogeneous computing resources such as high-performance computing platforms, in-vehicle computing systems, edge nodes, and cloud media processing nodes. The system can allocate and prioritize computing resources based on task computation intensity and latency sensitivity, guiding non-urgent tasks to migrate to lower-load periods while ensuring real-time performance and stable operation of critical services. This effectively alleviates peak computing power pressure and improves overall resource utilization efficiency. By implementing on-demand allocation and dynamic mapping strategies in different application scenarios, this invention avoids localized computing power overload, fully releases the potential of idle computing power, and enhances the continuity and reliability of various types of computing services while reducing computing power deployment and maintenance costs.

[0116] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0117] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0118] In this document, the singular forms “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that terms such as “comprising / including” or “having” specify the presence of the stated features, integrals, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integrals, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0119] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0120] The above description of the disclosed embodiments will enable those skilled in the art to make or use various modifications to these embodiments. It will be readily apparent to those skilled in the art that the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic resource allocation system based on computing power reservation and container mapping, characterized in that: It includes a time-segmentation module, a time-segment matching module, a computing power scheduling module, and a computing power optimization module. The functions of each module are as follows: Before a user submits a batch transcoding task, the time period segmentation module retrieves historical load data from the computing power cluster to analyze the computing power reserved time period, identifies peak computing power periods and off-peak computing power periods based on the computing power reserved time period, and passes them to the matching time period module. In the time period segmentation module, before the user submits a batch transcoding task, the statistical period is preset and divided into multiple statistical intervals; Within a preset statistical period, historical load data of the computing power cluster is retrieved through the computing power monitoring interface. The historical load data includes CPU utilization and task queue length. After standardizing the CPU utilization and task queue length, we obtain the CPU utilization coefficient and the task queue coefficient. If the CPU utilization coefficient is greater than the preset CPU utilization threshold within the statistical interval, the statistical interval is marked. Conversely, the statistical interval is classified as an off-peak period. The average value of the task queue is obtained by averaging the task queue coefficients within each statistical interval. If the task queue coefficient within the marked statistical interval is greater than the task queue mean, then the category of the marked statistical interval is determined to be a peak period. Conversely, the statistical interval is classified as an off-peak period. After obtaining the category results for each statistical interval, adjacent statistical intervals with consistent category results are merged to obtain peak computing power periods or off-peak computing power periods. Within a preset statistical period, the off-peak computing power period and peak computing power period of the day are vertically compared and overlapped on the time axis to count the frequency of occurrence of category results for the same period on different dates; Adjust peak computing power periods or off-peak computing power periods based on the frequency of occurrence; The matching time period module performs video attribute parsing and marks the transcoding category for batch transcoding tasks, accesses the category mapping database to match the target computing power time period corresponding to the transcoding category, detects the video frame rate of each transcoding task, and allocates different computing power time periods to the transcoding tasks in combination with the target computing power time period and passes them to the computing power scheduling module. In the matching time period module, the video attributes of the batch transcoding tasks are parsed through the video parsing interface to obtain the video file attribute information of each transcoding task, including the target transcoding specification and encoding format; Input the target transcoding specification and encoding format into the preset format mapping table, and output the transcoding category of the transcoding task; The access category mapping database matches the target computing power time period corresponding to the transcoding category. The target computing power time period includes the priority scheduling time period and the delayed execution time period. Preset basic weights for priority scheduling periods and delayed execution periods respectively; The video frame rate of each transcoding task is obtained through the video parsing interface. After the video frame rate of the transcoding task is standardized, it is multiplied by the basic weight corresponding to the transcoding task to obtain the time period adaptation value. If the time period adaptation value is greater than the preset time period adaptation threshold, the transcoding task will be assigned to the peak computing power period. Conversely, transcoding tasks will be allocated to off-peak computing time periods; When the computing power reservation period is in effect, the computing power scheduling module performs resource mapping processing on transcoding tasks, detects the task throughput of different computing power periods and calculates the task completion rate, counts the number of transcoding tasks corresponding to different computing power periods and evaluates the resource occupancy characteristics, and transmits the task completion rate and resource occupancy characteristics to the computing power optimization module. After receiving resource usage characteristics and task completion rates, the computing power optimization module generates a computing power adaptation score. Based on the computing power adaptation score, it filters and marks computing power time periods, selects transcoding tasks to be processed within the marked computing power time periods, and determines whether to change the computing power time period.

2. The dynamic resource regulation system based on computing power reservation and container mapping according to claim 1, characterized in that: In the computing power scheduling module, when the computing power reservation period is in effect, it receives the transcoding tasks and their corresponding computing power periods output by the matching period module, and performs resource mapping processing on each transcoding task according to the available resource status of each node in the computing power cluster. After completing the resource mapping process, the number of video frames completed by each transcoding task in each computing power period is counted and used as the task throughput of each transcoding task in different computing power periods. The total frame processing volume of all transcoding tasks during the computing power period is obtained by summing up the frame processing volume of all transcoding tasks during the computing power period. The ratio of task throughput to total frame processing volume is used as the task completion rate.

3. The dynamic resource regulation system based on computing power reservation and container mapping according to claim 1, characterized in that: In the computing power scheduling module, all transcoding tasks currently in execution are extracted from the task scheduling record. The transcoding tasks are grouped and statistically analyzed according to the computing power time period identifier to which the transcoding tasks belong, so as to obtain the number of transcoding tasks corresponding to each computing power time period. The maximum number of tasks to run is retrieved from the resource configuration database. The maximum number of tasks to run is the maximum number of tasks that the cluster can run simultaneously under the current configuration conditions. The ratio of the number of transcoding tasks to the maximum number of tasks can be used as a resource consumption characteristic.

4. The dynamic resource regulation system based on computing power reservation and container mapping according to claim 3, characterized in that: In the computing power optimization module, after receiving the resource occupancy characteristics and task completion rate, they are standardized to obtain the resource occupancy factor and task rate factor. The resource occupancy factor and task rate factor are calculated comprehensively using a weighted summation algorithm to obtain a computing power matching score. If the computing power adaptation score is greater than the preset adaptation score threshold, the computing power period is marked. Conversely, if the computing power matching score is less than or equal to the preset matching score threshold, the computing power period will not be marked.

5. The dynamic resource regulation system based on computing power reservation and container mapping according to claim 1, characterized in that: In the computing power optimization module, for the marked computing power period, the transcoding tasks in the execution state are retrieved from the task scheduling record to obtain the transcoding tasks to be processed, and the task throughput of the transcoding tasks to be processed in the marked computing power period is detected. The average throughput of transcoding tasks across all computing power periods is used as the baseline throughput. If the throughput of the transcoding task to be processed is less than the baseline throughput, the transcoding task to be processed will be marked as a portable task, and a computing power time period change instruction will be generated.

Images