Streaming media model computing resource allocation method based on LSTF algorithm

Abstract

The invention relates to a streaming media model computing resource allocation method based on an LSTF algorithm, and belongs to the technical field of computing resource prediction, and the method comprises the following steps: S1, obtaining video streaming parameters through real-time collection and historical data integration, and carrying out testing through a reference server to obtain a reference resource utilization rate; s2, based on the video stream parameters and the resource utilization rate, performing model training by using an Informer neural network model based on an LSTF algorithm to obtain a resource prediction model; s3, for different production scene requirements, defining a resource redundancy proportion by a user through resource redundancy factors in the resource prediction model, and predicting and calculating a resource utilization rate by using the resource prediction model; s4, automatically carrying out resource capacity expansion according to the comparison between the predicted resource utilization rate and the actual resource utilization rate; and S5, continuously collecting actual resource utilization rate data of the streaming media model, repeatedly training the resource prediction model according to a period, and optimizing model prediction precision.

Description

Technical Field

[0001] The invention belongs to the technical field of computing resource prediction and relates to a streaming media model computing resource allocation method based on LSTF algorithm. Background Art

[0002] With the rapid development of computer vision and video processing technology, streaming models are widely used in the industrial field for quality inspection, production monitoring, defect detection and dimensional measurement. However, these models usually require a lot of computing resources, especially when processing high-resolution and high-frame-rate video streams. Under the condition of limited hardware resources, how to effectively allocate resources to ensure the computing performance of the model becomes an important issue.

[0003] Conventional methods often use fixed resource allocation and issue alarms to expand capacity when resource usage exceeds the threshold. However, this method has a certain lag, resulting in low resource utilization or unstable model performance, and cannot dynamically make predictive adjustments based on actual video stream parameters. Therefore, a method is needed that can automatically predict and dynamically allocate computing resources in advance. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a streaming media model computing resource allocation method based on a temporal neural network model of an LSTF (Long Sequence Time-Series Forecasting) algorithm combined with device video stream parameters.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for allocating computing resources of a streaming media model based on an LSTF algorithm comprises the following steps:

[0007] S1: Obtain video stream parameters through real-time acquisition and historical data integration, and test through the benchmark server to obtain benchmark resource utilization;

[0008] S2: Based on the video stream parameters and resource usage rate, the Informer neural network model based on the LSTF algorithm is used to perform model training to obtain a resource prediction model;

[0009] S3: To meet the universal and differentiated requirements of different production scenarios, the resource redundancy factor in the resource prediction model is used to define the resource redundancy ratio and use the resource prediction model to predict the computing resource utilization rate.

[0010] S4: Automatically expand resources based on the comparison between predicted resource utilization and actual resource utilization;

[0011] S5: Continuously collect the actual resource usage data of the streaming media model, repeatedly train the resource prediction model on a periodic basis, and automatically optimize the model prediction accuracy.

[0012] Furthermore, the video stream parameters in step S1 include bit rate b, resolution r, frame rate f, I frame interval i and encoding type e; the benchmark resource utilization rate includes the CPU utilization ratio CPU_u, GPU utilization ratio GPU_u and memory utilization ratio Mem_u of the benchmark server.

[0013] Furthermore, step S1 also includes preprocessing the acquired video stream parameters and benchmark resource usage, including feature extraction, normalization processing and missing value completion processing.

[0014] Furthermore, in step S2, the Informer neural network based on the LSTF algorithm takes the bit rate b, resolution r, frame rate f, I frame interval i and encoding type e as input features, and takes the CPU usage ratio CPU_u, GPU usage ratio GPU_u and memory usage ratio Mem_u as target variables to train the resource prediction model.

[0015] Furthermore, in step S3, the trained model is used to predict the required computing resources according to the input video stream parameters; a resource redundancy factor is introduced and the redundancy ratio is customized; the resource allocation is calculated as follows:

[0016] CPU=CPU_b×CPU_u / 100×(1+CPU_r)

[0017] GPU=GPU_b×GPU_u / 100×(1+GPU_r)

[0018] MEM=Mem_b×Mem_u / 100×(1+Men_r)

[0019] Among them, CPU_b, GPU_b, and Mem_b are the CPU, GPU, and memory configurations of the benchmark server respectively; CPU_u, GPU_u, and Mem_u are the predicted CPU, GPU, and memory resource utilization rates respectively; CPU_r, GPU_r, and Mem_r are the resource CPU, GPU, and memory redundancy ratios respectively.

[0020] Further, in step S4, when the difference between the actual monitoring data and the predicted data is less than the redundancy ratio, the resource scheduling module is used to expand the resources.

[0021] Furthermore, in step S5, the actual resource usage data of the streaming media model is continuously collected based on the Prometheus monitoring system; combined with the custom script, the resource prediction model is automatically trained repeatedly on a periodic basis, and training and tuning are performed on a daily, weekly or monthly basis. The model parameter weights are adjusted regularly to optimize the model generalization ability and adapt to changes in video streaming parameters and computing resource requirements.

[0022] The beneficial effects of the present invention are as follows: the present invention can predict the required computing resources according to the parameters such as the bit rate, resolution, frame rate, encoding type, etc. of the video stream, automatically expand the resources in advance, avoid the occurrence of failures, improve the operating performance of the streaming media model and reduce the risk of failures. At the same time, a resource redundancy factor is introduced into the prediction model, and the user can define the resource redundancy ratio to meet the general and differentiated requirements of different production scenarios. The model can automatically repeat training regularly based on historical data, adaptively optimize the model generalization ability and improve the prediction accuracy.

[0023] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0025] Figure 1 A schematic diagram of the system modules used in the method for allocating computing resources for a streaming media model based on the LSTF algorithm;

[0026] Figure 2 The flowchart of the resource allocation method for the streaming media model calculation based on the LSTF algorithm;

[0027] Figure 3 This is the system data flow diagram used by the streaming media model computing resource allocation method based on the LSTF algorithm. DETAILED DESCRIPTION

[0028] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0029] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0030] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0031] like Figure 1-3 As shown, the present invention provides a streaming media model computing resource allocation method based on LSTF algorithm, comprising the following steps:

[0032] 1. Run the streaming media model on the benchmark configuration server, test different video streaming parameters, record the basic resource usage, and collect historical data on actual resource usage in production through real-time collection and historical data integration.

[0033] a) Baseline server configuration: baseline CPU (CPU_b), baseline memory (Mem_b), GPU (GPU_b).

[0034] b) Video stream parameters: b: bit rate (kbpsr), resolution (resolution), frame rate (framerate), I-frame interval (I-frame interval), encoding type (encoding type).

[0035] c) Resource usage during model calculation: CPU_u = CPU usage ratio, GPU_u = GPU usage ratio, Mem_u = memory usage ratio.

[0036] 2. Preprocess the collected data, including:

[0037] a) Feature extraction

[0038] b) Normalization

[0039] c) Missing value filling process.

[0040] 3. Based on the Informer time series prediction neural network model, this model is an LSTF (Long Sequence Time-Series Forecasting) algorithm, which is trained using the PyTorch deep learning framework. Informer introduces the ProbSparse Attention mechanism based on the Transformer model, which enhances the speed and efficiency of processing long sequences and improves the accuracy and stability of prediction.

[0041] a) Input features: bit rate b, resolution r, frame rate f, I frame interval i, encoding type e.

[0042] b) Target variables: CPU_u, GPU_u, Mem_u.

[0043] c)y^=f(B,R,F,I,E), where y^ represents the predicted resource usage and function f represents the trained prediction model function.

[0044] 4. Use the trained model to predict the required computing resources based on the input video stream parameters. Introduce resource redundancy factors, and users can define the redundancy ratio.

[0045] a)CPU_u,GPU_u,Mem_u=f(B,R,F,I,E)

[0046] b) Resource allocation calculation:

[0047] CPU=CPU_b×CPU_u / 100×(1+CPU_r)

[0048] GPU=GPU_b×GPU_u / 100×(1+GPU_r)

[0049] MEM=Mem_b×Mem_u / 100×(1+Men_r)

[0050] CPU_b, GPU_b, Mem_b are the CPU, GPU, and memory configurations of the benchmark server; CPU_u, GPU_u, Mem_u are the predicted CPU, GPU, and memory resource utilization rates; CPU_r, GPU_r, Mem_r are the resource CPU, GPU, and memory redundancy ratios.

[0051] 5. When the difference between the actual monitoring data and the predicted data is less than the redundancy ratio, use the resource scheduling module to expand resources. If the streaming media model uses a cloud platform such as Huawei Cloud or Alibaba Cloud, this can be achieved through the platform's elastic scaling function. If it is deployed on a common server, this can be achieved through a custom python script.

[0052] 6. Based on the data collection module (Prometheus monitoring system), the actual resource usage data of the streaming media model is continuously collected. Combined with custom scripts, the resource prediction model is automatically trained repeatedly in cycles. Training and tuning can be performed on a daily, weekly, and monthly basis, and the model parameter weights can be adjusted regularly to optimize the model's generalization ability and adapt to changes in video streaming parameters and computing resource requirements.

[0053] In the above embodiments, the description's reference to "this embodiment" indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0054] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0055] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0056] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0057] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0058] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0059] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0060] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0061] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0062] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0063] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for allocating computing resources for a streaming media model based on an LSTF algorithm, characterized in that: The following steps are involved: S1: Obtain video stream parameters through real-time acquisition and historical data integration, and test through the benchmark server to obtain benchmark resource utilization; S2: Based on the video stream parameters and resource usage rate, the Informer neural network model based on the LSTF algorithm is used to perform model training to obtain a resource prediction model; S3: To meet the universal and differentiated requirements of different production scenarios, the resource redundancy factor in the resource prediction model is used to define the resource redundancy ratio and use the resource prediction model to predict the computing resource utilization rate. S4: Automatically expand resources based on the comparison between predicted resource utilization and actual resource utilization; S5: Continuously collect the actual resource usage data of the streaming media model, repeatedly train the resource prediction model on a periodic basis, and automatically optimize the model prediction accuracy.

2. The method for allocating computing resources of a streaming media model based on the LSTF algorithm according to claim 1, characterized in that: The video stream parameters in step S1 include bit rate b, resolution r, frame rate f, I frame interval i and encoding type e; the benchmark resource utilization rate includes the CPU utilization ratio CPU_u, GPU utilization ratio GPU_u and memory utilization ratio Mem_u of the benchmark server.

3. A streaming media model computing resource allocation method based on LSTF algorithm, characterized by: Step S1 also includes preprocessing the acquired video stream parameters and benchmark resource usage rate. Including feature extraction, normalization and missing value filling.

4. A method for allocating computing resources for a streaming media model based on the LSTF algorithm, characterized in that: In step S2, the Informer neural network based on the LSTF algorithm takes the bit rate b, resolution r, frame rate f, I frame interval i and encoding type e as input features, and the CPU usage ratio CPU_u, GPU usage ratio GPU_u and memory usage ratio Mem_u as target variables to train the resource prediction model.

5. A method for allocating computing resources for a streaming media model based on the LSTF algorithm, characterized in that: In step S3, the trained model is used to predict the required computing resources based on the input video stream parameters; a resource redundancy factor is introduced and the redundancy ratio is customized; the resource allocation calculation is as follows: CPU=CPU_b×CPU_u / 100×(1+CPU_r) GPU=GPU_b×GPU_u / 100×(1+GPU_r) MEM=Mem_b×Mem_u / 100×(1+Men_r) Among them, CPU_b, GPU_b, and Mem_b are the CPU, GPU, and memory configurations of the benchmark server respectively; CPU_u, GPU_u, and Mem_u are the predicted CPU, GPU, and memory resource utilization rates respectively; CPU_r, GPU_r, and Mem_r are the resource CPU, GPU, and memory redundancy ratios respectively.

6. A method for allocating computing resources for a streaming media model based on the LSTF algorithm, characterized in that: In step S4, when the difference between the actual monitoring data and the predicted data is less than the redundancy ratio, the resource scheduling module is used to expand the resources.

7. A streaming media model computing resource allocation method based on LSTF algorithm, characterized by: In step S5, the actual resource usage data of the streaming media model is continuously collected based on the Prometheus monitoring system; combined with the custom script, the resource prediction model is automatically trained repeatedly on a periodic basis, and training and tuning are performed on a daily, weekly or monthly basis. The model parameter weights are adjusted regularly to optimize the model generalization ability and adapt to changes in video streaming parameters and computing resource requirements.

Images