Adaptive cloud computer video stream concurrent processing scheduling and storage method and system
By combining real-time monitoring and predictive analytics with dynamic scheduling and storage optimization, the problems of video data management and scheduling in cloud computing environments have been solved, achieving efficient resource utilization and rapid response, and reducing energy consumption and operating costs.
Patent Information
- Application Number
- CN202511121413.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-12
- Publication Date
- 2025-11-07
AI Technical Summary
In cloud computing environments, issues such as video data loss, slow page loading, lag, and request timeouts negatively impact user experience, and existing technologies struggle to effectively manage and schedule concurrent tasks in cloud computers.
By monitoring cloud environment resource usage in real time and combining user requests and task types, resource allocation is dynamically adjusted; a hybrid prediction algorithm combining ARIMA time series analysis and LSTM neural network is used to predict concurrent request volume; dynamic scheduling and storage optimization are implemented, a cold and hot data hierarchical model is established, resource reservation and elastic scaling are performed, and computing and storage strategies are optimized.
It improves resource utilization, reduces energy consumption and operating costs, ensures fast response and low latency for concurrent processing of cloud computer video streams, and meets the scheduling needs of different tasks.
Smart Images
Figure CN120909797A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of machine learning prediction, in particular to a method and system for adaptive cloud computer video stream concurrent processing scheduling and storage. BACKGROUND
[0002] With the rapid development and wide application of cloud computing technology, more and more enterprises and individuals begin to use cloud computers to store and process data in order to improve efficiency and reduce cost. As a new type of computing mode, cloud computer greatly expands the scale and range of computing resources and tasks, and can flexibly allocate computing resources. However, with the increase of the number and complexity of tasks in the cloud environment, it may cause the loss of obtained video data, slow page loading, lag, request timeout, and even page crash, affecting user use. Therefore, how to effectively manage and schedule these tasks has become a problem to be solved. SUMMARY
[0003] The technical task of the present application is to solve the above problems, and provide a method and system for adaptive cloud computer video stream concurrent processing scheduling and storage. Through real-time monitoring of cloud computer video resources and usage, intelligent scheduling of concurrent tasks is performed, thereby improving resource utilization, reducing loss and operating cost.
[0004] The technical scheme adopted by the present application to solve its technical problems is:
[0005] A method for adaptive cloud computer video stream concurrent processing scheduling and storage, the implementation of the method includes the following steps:
[0006] Real-time monitoring: real-time load monitoring is performed, and the resource usage of each node in the cloud environment is obtained in real time through the monitoring system, and the resource allocation is dynamically adjusted in combination with the concurrent quantity and task type of user requests;
[0007] Predictive analysis: collect and analyze historical video stream request data, identify peak periods and popular video content; based on historical load data, establish a time series prediction model to predict the concurrent request quantity and resource demand in a future set time window;
[0008] Dynamic scheduling: according to real-time monitoring data and prediction results, implement preprocessing acceleration strategy for computing-intensive tasks, and dynamically allocate computing resources by using weight priority algorithm;
[0009] Storage optimization: by analyzing data access patterns to establish a hot and cold data grading model, hot data with high frequency access is stored in a high-speed storage layer, and cold data with low frequency is migrated to a low-cost storage layer; hot video content with high frequency access is stored in cache to reduce disk I / O delay and speed up loading; cold data that is not frequently accessed is stored in low-cost storage media to reduce storage cost.
[0010] Elastic adjustment: according to the predicted concurrent request fluctuation value, the deployment scale of virtual machine and the capacity allocation of resource pool are dynamically adjusted through the preset threshold triggering rule.
[0011] The method can improve the utilization rate of resources, reduce energy consumption and operating cost, and better meet the scheduling needs of cloud computer concurrency.
[0012] Further, the resource usage of each node in the cloud environment includes CPU usage, memory occupancy, network bandwidth, and storage I / O indicators of the cloud computer node.
[0013] Further, the prediction analysis uses a hybrid prediction algorithm combining ARIMA time series analysis and LSTM neural network, and the short-term prediction uses a sliding window mechanism.
[0014] Further, the short-term prediction uses a sliding window mechanism, and the prediction time granularity is 5-15 minutes.
[0015] Further, the compute-intensive task includes video transcoding task.
[0016] Further, the dynamic scheduling includes a resource reservation system that preallocates 10%-20% of redundant computing resources when a burst load is predicted and establishes a virtual machine rapid cloning template library.
[0017] Further, the storage optimization sets a dynamic migration strategy that automatically downgrades the storage hierarchy when the continuous access interval of data blocks exceeds the set threshold, and triggers the data prefetching mechanism when the access frequency reaches the upgrade threshold.
[0018] The application also claims a system for adaptive cloud computer video stream concurrency processing and scheduling, comprising:
[0019] A real-time monitoring module is configured to continuously collect resource usage of each node in the cloud environment, including CPU usage, memory occupancy, network bandwidth, and storage I / O indicators, etc.
[0020] A prediction analysis module is configured to establish a time series prediction model based on historical load data, and to predict concurrent request volume and resource demand in a future set time window.
[0021] A dynamic scheduling module is configured to implement a pre-processing acceleration strategy for compute-intensive tasks according to real-time monitoring data and prediction results, and to dynamically allocate computing resources using a weight priority algorithm.
[0022] A storage optimization module is configured to establish a hot and cold data classification model by analyzing data access patterns, and to store frequently accessed hot data in a high-speed storage layer and to migrate low-frequency cold data to a low-cost storage layer.
[0023] An elastic scaling module is configured to dynamically adjust the deployment scale of virtual machines and the capacity allocation of resource pools according to predicted concurrent request fluctuation values and through a pre-set threshold triggering rule.
[0024] The system implements the above-mentioned adaptive cloud computer video stream concurrent processing scheduling and storage method.
[0025] The application also claims a kind of adaptive cloud computer video stream concurrent processing scheduling and storage device, comprising: at least one memory and at least one processor;
[0026] The at least one memory is configured to store machine-readable programs.
[0027] The at least one processor is configured to call the machine-readable programs to implement the above-mentioned method.
[0028] The application also claims a computer-readable medium having computer instructions stored thereon, which, when executed by a processor, can implement the above-mentioned method.
[0029] Compared with the prior art, the adaptive cloud computer video stream concurrent processing scheduling and storage method and system of the application has the following advantages:
[0030] The application realizes rapid response and dynamic adjustment through real-time monitoring and prediction analysis, improves prediction accuracy by combining statistical and machine learning prediction algorithms, realizes efficient resource utilization and performance optimization through elastic scaling strategies and load balancing techniques, automatically adjusts scheduling strategies according to real-time conditions to adapt to different task requirements and resource changes, avoids platform data loss and stuttering, and improves platform performance. BRIEF DESCRIPTION OF DRAWINGS
[0031] Figure 1is the method implementation principle diagram of adaptive cloud computer video stream concurrent processing scheduling and storage provided by the embodiment of the application. DETAILED DESCRIPTION
[0032] The application will be further described below in combination with specific embodiments.
[0033] The embodiment of the application provides a method for adaptive cloud computer video stream concurrent processing scheduling and storage, and the implementation of the method comprises the following steps:
[0034] 1. Real-time monitoring: Real-time load monitoring is performed, resource usage conditions such as CPU, memory, network bandwidth of the cloud computer node are acquired in real time through a monitoring system, and resource allocation is dynamically adjusted in combination with the concurrent quantity and task type of user requests.
[0035] 2. Predictive analysis: Historical video stream request data are collected and analyzed, peak periods and popular video contents are identified, a time series prediction model is established based on historical load data, and concurrent request quantities and resource demands in a future set time window are predicted.
[0036] 3. Dynamic scheduling: According to real-time monitoring data and prediction results, pretreatment acceleration strategies are implemented for computing-intensive tasks such as video transcoding, and a weight priority algorithm is used to dynamically allocate computing resources.
[0037] 4. Storage optimization: A hot and cold data grading model is established by analyzing data access patterns, hot data with high frequency access is stored in a high-speed storage layer, and cold data with low frequency is migrated to a low-cost storage layer; popular video contents with high frequency access are stored in a cache, disk I / O delay is reduced, and loading speed is accelerated; cold data with infrequent access is stored in a low-cost storage medium, and storage cost is reduced.
[0038] 5. Elastic scaling adjustment: According to predicted concurrent request fluctuation values, the deployment scale of a virtual machine and the capacity allocation of a resource pool are dynamically adjusted through preset threshold triggering rules.
[0039] The prediction analysis adopts a hybrid prediction algorithm combining ARIMA time series analysis and an LSTM neural network, wherein a sliding window mechanism is used for short-term prediction, and the prediction time granularity is 5-15 minutes.
[0040] The dynamic scheduling comprises a resource reservation system, 10%-20% of redundant computing resources are pre-allocated in advance when a burst load is predicted, and a virtual machine rapid cloning template library is established.
[0041] The storage optimization sets a dynamic migration strategy, when a continuous access interval of a data block exceeds a set threshold, a storage level is automatically downgraded, and when an access frequency reaches an upgrade threshold, a data prefetching mechanism is triggered.
[0042] The method can improve the utilization rate of resources, reduce energy consumption and operation cost, and better meet the scheduling needs of cloud computers.
[0043] The embodiment of the present application also provides a system for adaptive cloud computer video stream concurrent processing scheduling and storage.
[0044] 1. A real-time monitoring module for continuously collecting CPU usage, memory occupancy, network bandwidth and storage I / O indicators of each node in the cloud environment. And combined with the concurrent quantity and task type of user request, dynamically adjust resource allocation.
[0045] 2. A prediction analysis module for collecting and analyzing historical video stream request data, identifying peak periods and popular video content; based on historical load data, a time series prediction model is established to predict the concurrent request quantity and resource demand in the future set time window.
[0046] 3. A dynamic scheduling module for implementing pre-processing acceleration strategy for computing-intensive tasks according to real-time monitoring data and prediction results, and dynamically allocating computing resources using weight priority algorithm.
[0047] 4. A storage optimization module for establishing a hot and cold data classification model by analyzing data access patterns, storing high-frequency access hot data in a high-speed storage layer, and migrating low-frequency cold data to a low-cost storage layer.
[0048] Storing popular video content with high frequency access in cache can reduce disk I / O delay and speed up loading speed; storing cold data with low frequency access in low-cost storage medium can reduce storage cost.
[0049] 5. An elastic scaling module for dynamically adjusting the deployment scale of virtual machines and the capacity allocation of resource pools according to the predicted concurrent request fluctuation value through the preset threshold triggering rule.
[0050] The prediction analysis module uses a hybrid prediction algorithm combining ARIMA time series analysis and LSTM neural network, where short-term prediction uses a sliding window mechanism with a prediction time granularity of 5-15 minutes.
[0051] The dynamic scheduling module comprises a resource reservation subsystem, which pre-allocates 10%-20% of redundant computing resources in advance when a burst load is predicted, and establishes a virtual machine fast cloning template library.
[0052] The storage optimization module sets a dynamic migration strategy, automatically degrades the storage level when the data block continuous access interval exceeds a set threshold, and triggers a data prefetch mechanism when the access frequency reaches an upgrade threshold.
[0053] The embodiment of the application also provides a device for adaptive cloud computer video stream concurrent processing scheduling and storage, comprising at least one memory and at least one processor.
[0054] The at least one memory is used for storing machine readable programs.
[0055] The at least one processor is used for calling the machine readable programs, and realizing the method for adaptive cloud computer video stream concurrent processing scheduling and storage described in the above embodiment.
[0056] The embodiment of the application also provides a computer readable medium, wherein the computer readable medium stores computer instructions, and the computer instructions realize the method for adaptive cloud computer video stream concurrent processing scheduling and storage described in the above embodiment when executed by a processor.
[0057] In this case, the program code read from the storage medium can realize the function of any one of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the application.
[0058] The storage medium for providing the program code includes a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a nonvolatile memory card and a ROM.
[0059] In addition, it should be clear that not only the program code read by the computer can be executed, but also part or all of the actual operations can be completed by the operating system and the like operating on the computer based on the instructions of the program code, so as to realize the function of any one of the above embodiments.
[0060] Further, it is understood that the programs read out from the storage medium can be written to the memories of an expansion board inserted into the computer or an expansion unit connected to the computer, and then, on the basis of instructions of the programs, the CPU or the like installed on the expansion board or the expansion unit can execute some or all of the actual operations, thereby implementing the functions of any of the above-described embodiments.
[0061] The present application has been described in detail by the above drawings and preferred embodiments, however, the present application is not limited to these disclosed embodiments, and as can be known by those skilled in the art based on the above-described embodiments, more embodiments of the present application can be obtained by combining the code review means in the above-described different embodiments, and these embodiments are also within the protection scope of the present application.
Claims
1. A method for adaptive cloud computer video stream concurrent processing scheduling and storage, characterized in that, The implementation of the method comprises the following steps: Real-time monitoring: Real-time load monitoring is performed to obtain the resource usage of each node in the cloud environment in real time through the monitoring system, and the resource allocation is dynamically adjusted in combination with the concurrency of user requests and the task type; Predictive analysis: Collect and analyze historical video stream request data to identify peak periods and popular video content; based on historical load data, a time series prediction model is established to predict the concurrent request volume and resource demand within a future set time window; Dynamic scheduling: Based on real-time monitoring data and prediction results, implement preprocessing acceleration strategies for compute-intensive tasks, and dynamically allocate computing resources using a weight priority algorithm; Storage optimization: By analyzing data access patterns, a hot and cold data classification model is established to store frequently accessed hot data in a high-speed storage layer and migrate low-frequency cold data to a low-cost storage layer; Elastic scaling adjustment: According to the predicted concurrent request fluctuation value, dynamically adjust the deployment scale of virtual machines and the capacity allocation of resource pools through pre-set threshold triggering rules. 2.The method of claim 1, wherein, The resource usage of each node in the cloud environment includes CPU usage, memory occupancy, network bandwidth, and storage I / O indicators of the cloud computer node. 3.The method of claim 1, wherein, The predictive analysis uses a hybrid prediction algorithm combining ARIMA time series analysis and LSTM neural networks, with a sliding window mechanism for short-term prediction.
4. The method of adaptive cloud computer video stream concurrent processing, scheduling and storage according to claim 3, characterized in that, The short-term prediction uses a sliding window mechanism with a prediction time granularity of 5-15 minutes.
5. The method of adaptive cloud computer video stream concurrent processing, scheduling and storage of claim 1, wherein, The dynamic scheduling includes a resource reservation system that pre-allocates 10%-20% of redundant computing resources in advance when a sudden load is predicted and establishes a virtual machine rapid cloning template library.
6. The method of adaptive cloud computer video stream concurrent processing, scheduling and storage of claim 1, wherein, The compute-intensive tasks include video transcoding tasks.
7. The method of adaptive cloud computer video stream concurrent processing, scheduling and storage of claim 1, wherein, The storage optimization sets a dynamic migration strategy that automatically downgrades the storage level when the continuous access interval of data blocks exceeds the set threshold and triggers the data prefetching mechanism when the access frequency reaches the upgrade threshold.
8. A system for adaptive cloud computer video stream concurrent processing scheduling and storage, characterized in that, It includes: A real-time monitoring module for continuously collecting resource usage of each node in the cloud environment; A predictive analysis module that establishes a time series prediction model based on historical load data to predict the concurrent request volume and resource demand within a future set time window; A dynamic scheduling module that implements preprocessing acceleration strategies for compute-intensive tasks based on real-time monitoring data and prediction results, and dynamically allocates computing resources using a weight priority algorithm; A storage optimization module that establishes a hot and cold data classification model by analyzing data access patterns to store frequently accessed hot data in a high-speed storage layer and migrate low-frequency cold data to a low-cost storage layer; An elastic scaling module that dynamically adjusts the deployment scale of virtual machines and the capacity allocation of resource pools based on the predicted concurrent request fluctuation value through pre-set threshold triggering rules. The system implements the method of any one of claims 1-7.
9. An apparatus for adaptive cloud computer video stream concurrent processing scheduling and storage, characterized in that, It includes: At least one memory and at least one processor; The at least one memory is used to store machine-readable programs; The at least one processor is used to call the machine-readable programs to implement the method of any one of claims 1-7.
10. A computer readable medium characterized by The computer readable medium stores computer instructions which, when executed by the processor, enable the method of any one of claims 1 to 7.