Data compression tool dynamic scheduling method and system based on computing power network

Abstract

The application discloses a data compression tool dynamic scheduling method and system based on a computing power network, belongs to the technical field of cloud computing, and aims to solve the technical problems of combining the computing power network with data compression tool scheduling of a computing power terminal and improving the data compression efficiency of the computing power terminal. The method comprises the following steps: collecting performance index data of the computing power terminal through monitoring of a client, and reporting the performance index data to a data compression tool scheduler at regular time intervals; scheduling the data compression tool of the computing power terminal based on the performance index data through the data compression tool scheduler, wherein, during the scheduling, the matching degree of the computing power terminal and the data compression tool is calculated, and the matched data compression tool is selected from a data compression tool library deployed on the computing power terminal according to the matching degree; and real-time monitoring of resource state data is performed, and when the resource state deteriorates, the data resource scheduling tool is triggered to perform re-scheduling.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, specifically to a dynamic scheduling method and system for data compression tools based on computing power networks. Background Technology

[0002] Data compression tools are software tools used to reduce the size of files or data for storage, transmission, or sharing. They employ compression algorithms to remove or optimize redundant information in file content, significantly reducing storage space without sacrificing original data quality. These tools are widely used in various scenarios, such as network transmission, data backup, and storage space optimization, providing users with convenience and cost savings by offering efficient compression and decompression capabilities. Computing networks are an indispensable new type of information infrastructure in the era of intelligence. Through cloud-network convergence technology, they unify the scheduling and management of distributed computing, storage, and network resources, enabling on-demand allocation and flexible scheduling of computing resources. Computing networks can support various application scenarios, from scientific computing to data processing, from image processing to virtual reality, providing efficient and reliable computing power support and a strong foundation for digital transformation and intelligent upgrading.

[0003] As critical nodes for processing and storing data, computing terminals often face challenges in data compression due to resource constraints and the complexity of network environments. These challenges include low efficiency and slow processing speeds, impacting both the real-time nature of data transmission and limiting the terminal's computational potential. Traditional data compression typically employs fixed compression tools, focusing primarily on optimizing the compression algorithm itself, such as improving compression ratio and speed. However, these algorithms often neglect the actual conditions of the computing terminal, such as CPU and memory utilization, and network conditions like uplink / downlink speeds and packet loss rates. Under resource constraints, even highly efficient compression algorithms may fail to achieve optimal performance due to insufficient resources, unable to dynamically adjust to changing resource conditions and network environments. Furthermore, with the rise of cloud computing and edge computing, the concept of computing networks has become a research hotspot. Computing networks aim to provide more powerful and flexible computing capabilities through the collaborative work of distributed computing resources.

[0004] The technical problems that need to be solved are how to combine the scheduling of computing power networks with data compression tools on computing power terminals and how to improve the data compression efficiency of computing power terminals. Summary of the Invention

[0005] The technical objective of this invention is to address the above-mentioned shortcomings by providing a dynamic scheduling method and system for data compression tools based on computing power networks, thereby solving the computational problems of combining computing power networks with the scheduling of data compression tools on computing power terminals and improving the data compression efficiency of computing power terminals.

[0006] In a first aspect, the present invention provides a dynamic scheduling method for data compression tools based on a computing power network, applied to a computing power terminal cluster. Multiple computing power terminals with computing power resources and network connectivity are combined into a collaborative computing power terminal cluster. Each computing power terminal is equipped with a monitoring client, and the computing power terminal cluster is equipped with a data compression tool scheduler. The method includes the following steps:

[0007] Data Acquisition: The performance index data of the computing terminal is collected through the monitoring client, and the performance index data is reported to the data compression tool scheduler at regular intervals. The performance index data includes resource status data and network environment data.

[0008] Matching analysis: Based on performance index data, the data compression tools of the computing power terminal are scheduled through the data compression tool scheduler. During scheduling, the matching degree between the computing power terminal and the data compression tool is calculated, and a matching data compression tool is selected from the data compression tool library deployed on the computing power terminal according to the matching degree.

[0009] Resource monitoring: Real-time monitoring of resource status data; when resource conditions deteriorate, triggering the data resource scheduling tool for rescheduling.

[0010] Preferably, the data compression tool of the computing terminal is scheduled based on performance index data through a data compression tool scheduler, including the following steps:

[0011] Obtain the data compression tool library from the computing power terminal through the data compression tool scheduler;

[0012] Establish a performance baseline for each data compression tool, including performance under ideal conditions, performance under general conditions, and performance under resource-constrained conditions;

[0013] Based on the performance index data of the previous time series, the performance index data of the next time series is predicted by the data compression tool scheduler to obtain the predicted value of the performance index data;

[0014] Based on the predicted values ​​of performance index data and the performance baseline of data compression tools, a matching score is calculated for each data compression tool in the computing terminal data compression tool library according to the degree of matching between the resource consumption of the data compression tool and the current resource availability.

[0015] Compare the matching scores of different data compression tools, and select and schedule the most suitable data compression tool from the data compression tool library of the computing power terminal based on the matching score.

[0016] As a preferred method, the performance index data of the next time series is predicted based on the performance index data of the previous time series and using a simple exponential smoothing time series forecasting method.

[0017] Preferably, resource status data includes CPU utilization and memory utilization, and network environment data includes uplink and downlink speeds, packet loss rate, and jitter.

[0018] Secondly, the present invention provides a dynamic scheduling system for data compression tools based on computing power networks, which is applied to computing power terminal clusters. Multiple computing power terminals with computing power resources and network connection functions are combined into a collaborative computing power terminal cluster. The system includes a monitoring client, a data compression tool scheduler, and a resource monitoring module.

[0019] The monitoring client is used to collect performance index data of computing terminals and periodically report the performance index data to the data compression tool scheduler. The performance index data includes resource status data and network environment data.

[0020] The data compression tool scheduler is used to schedule data compression tools on computing power terminals based on performance index data. During scheduling, it calculates the matching degree between computing power terminals and data compression tools, and selects a matching data compression tool from the data compression tool library deployed on the computing power terminals according to the matching degree.

[0021] The resource monitoring module is used to monitor resource status data in real time. When the resource status deteriorates, it triggers the data resource scheduling tool to reschedule.

[0022] Preferably, the data compression tool scheduler is used to schedule the data compression tools on the computing terminal as follows:

[0023] Obtain a data compression tool library for computing power terminals;

[0024] Establish a performance baseline for each data compression tool, including performance under ideal conditions, performance under general conditions, and performance under resource-constrained conditions;

[0025] Based on the performance index data of the previous time series, the performance index data of the next time series is predicted to obtain the predicted value of the performance index data.

[0026] Based on the predicted values ​​of performance index data and the performance baseline of data compression tools, a matching score is calculated for each data compression tool in the computing terminal data compression tool library according to the degree of matching between the resource consumption of the data compression tool and the current resource availability.

[0027] Compare the matching scores of different data compression tools, and select and schedule the most suitable data compression tool from the data compression tool library of the computing power terminal based on the matching score.

[0028] Preferably, based on the performance index data of the previous time series, the data compression tool scheduler is used to predict the performance index data of the next time series using a simple exponential smoothing time series forecasting method.

[0029] Preferably, resource status data includes CPU utilization and memory utilization, and network environment data includes uplink and downlink speeds, packet loss rate, and jitter.

[0030] The dynamic scheduling method and system for data compression tools based on computing power networks of the present invention have the following advantages:

[0031] 1. By collecting real-time data on the resource usage and network environment of computing terminals, the system dynamically adjusts and intelligently selects and schedules the most suitable data compression tools based on the real-time status of the computing terminals.

[0032] 2. When resources are scarce, choose a data compression tool that consumes fewer resources, and when resources are plentiful, choose a data compression tool with better compression effect. When resource conditions deteriorate, the ability to respond promptly and reschedule data compression tools can significantly improve data compression efficiency and shorten data compression time. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] The invention will be further described below with reference to the accompanying drawings.

[0035] Figure 1 This is a flowchart of a dynamic scheduling method for data compression tools based on computing power networks, as described in Example 1. Detailed Implementation

[0036] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0037] This invention provides a method and system for dynamic scheduling of data compression tools based on computing power networks, which solves the technical problem of combining the scheduling of computing power networks with the data compression tools of computing power terminals and how to improve the data compression efficiency of computing power terminals.

[0038] Example 1:

[0039] This invention discloses a dynamic scheduling method for data compression tools based on a computing power network. It is applied to a computing power terminal cluster, where multiple computing power terminals with computing resources and network connectivity are combined into a collaborative cluster. Each computing power terminal is equipped with a monitoring client, and the computing power terminal cluster is equipped with a data compression tool scheduler. The method includes three steps: data acquisition, matching analysis, and resource monitoring.

[0040] Step S100 Data Acquisition: Collect performance index data of computing terminals through the monitoring client, and report the performance index data to the data compression tool scheduler at regular intervals. The performance index data includes resource status data and network environment data.

[0041] In this embodiment, resource status data includes CPU utilization and memory utilization, and network environment data includes uplink and downlink speeds, packet loss rate, and jitter.

[0042] Step S200 Matching Analysis: Based on performance index data, the data compression tools of the computing power terminal are scheduled through the data compression tool scheduler. During scheduling, the matching degree between the computing power terminal and the data compression tool is calculated, and a matching data compression tool is selected from the data compression tool library deployed on the computing power terminal according to the matching degree.

[0043] As a specific implementation of the matching analysis, based on performance index data, the data compression tools of the computing terminals are scheduled through a data compression tool scheduler, including the following steps:

[0044] (1) Obtain the data compression tool library of the computing terminal through the data compression tool scheduler;

[0045] (2) Establish a performance baseline for each data compression tool, including performance under ideal conditions, performance under general conditions, and performance under resource-constrained conditions;

[0046] (3) Based on the performance index data of the previous time series, the performance index data of the next time series is predicted by the data compression tool scheduler to obtain the predicted value of the performance index data.

[0047] (4) Based on the predicted values ​​of performance index data and the performance baseline of data compression tools, calculate a matching score for each data compression tool in the computing terminal data compression tool library according to the matching degree between the resource consumption of the data compression tool and the current resource availability;

[0048] (5) Compare the matching scores of different data compression tools, and select and schedule the most suitable data compression tool from the data compression tool library of the computing terminal based on the matching scores.

[0049] This method involves forecasting the performance indicators of the next time series based on the performance indicator data of the previous time series using a simple exponential smoothing time series forecasting method. A smoothing coefficient α is pre-set; a smaller α can be selected when the time series exhibits a relatively stable horizontal trend, while a larger α can be selected when the time series fluctuates significantly. The predicted value of the next time series is predicted using the performance indicator values ​​(observed values) and predicted values ​​reported from the previous time series.

[0050] Step S300 Resource Monitoring: Monitor resource status data in real time. When the resource status deteriorates, trigger the data resource scheduling tool to reschedule.

[0051] In this embodiment, this step continuously monitors the resource status of the computing terminal. When the resource status is detected to be deteriorating, the data compression tool scheduler is triggered to reschedule.

[0052] This embodiment's method collects real-time data on the computing terminal's resource status (CPU utilization, memory utilization, etc.) and network environment (uplink / downlink speeds, packet loss rate, etc.). By analyzing the computing terminal's multi-dimensional performance indicators in real time, it intelligently schedules the most suitable data compression tool to the computing terminal. Furthermore, when the resource status on the computing terminal deteriorates, a more suitable data compression tool is rescheduled to that terminal. This method significantly improves data compression efficiency on the computing terminal side, thereby shortening data compression time.

[0053] Example 2:

[0054] This invention discloses a dynamic scheduling system for data compression tools based on computing power networks, comprising a monitoring client, a data compression tool scheduler, and a resource monitoring module.

[0055] The monitoring client is used to collect performance index data of computing terminals and periodically report the performance index data to the data compression tool scheduler. The performance index data includes resource status data and network environment data.

[0056] The resource status data includes CPU utilization and memory utilization, while the network environment data includes uplink and downlink speeds, packet loss rate, and jitter.

[0057] The data compression tool scheduler is used to schedule data compression tools on computing power terminals based on performance index data. During scheduling, it calculates the matching degree between computing power terminals and data compression tools, and selects a matching data compression tool from the data compression tool library deployed on the computing power terminals according to the matching degree.

[0058] In a specific implementation, the scheduler, based on performance metrics data and data compression tools, performs the following scheduling:

[0059] (1) Obtain the data compression tool library of the computing terminal through the data compression tool scheduler;

[0060] (2) Establish a performance baseline for each data compression tool, including performance under ideal conditions, performance under general conditions, and performance under resource-constrained conditions;

[0061] (3) Based on the performance index data of the previous time series, the performance index data of the next time series is predicted by the data compression tool scheduler to obtain the predicted value of the performance index data.

[0062] (4) Based on the predicted values ​​of performance index data and the performance baseline of data compression tools, calculate a matching score for each data compression tool in the computing terminal data compression tool library according to the matching degree between the resource consumption of the data compression tool and the current resource availability;

[0063] (5) Compare the matching scores of different data compression tools, and select and schedule the most suitable data compression tool from the data compression tool library of the computing terminal based on the matching scores.

[0064] This method involves forecasting the performance indicators of the next time series based on the performance indicator data of the previous time series using a simple exponential smoothing time series forecasting method. A smoothing coefficient α is pre-set; a smaller α can be selected when the time series exhibits a relatively stable horizontal trend, while a larger α can be selected when the time series fluctuates significantly. The predicted value of the next time series is predicted using the performance indicator values ​​(observed values) and predicted values ​​reported from the previous time series.

[0065] The resource monitoring module is used to monitor resource status data in real time. When the resource status deteriorates, it triggers the data resource scheduling tool to reschedule.

[0066] In this embodiment, the resource monitoring module is used to continuously monitor the resource status of the computing terminal. When the resource status is detected to be deteriorating, the data compression tool scheduler is triggered to reschedule.

[0067] The system in this embodiment can execute the method disclosed in Embodiment 1 to dynamically schedule the data compression tool for the terminal device.

[0068] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the means in the different embodiments described above, and these embodiments are also within the protection scope of the present invention.

Claims

1. A computing power network-based data compression tool dynamic scheduling method, characterized in that, The application is applied to a computing terminal cluster, multiple computing terminals with computing resources and network connection functions are combined into a cooperative computing terminal cluster, the computing terminal is deployed with a monitoring client, and the computing terminal cluster is deployed with a data compression tool scheduler, and the method comprises the following steps: Data acquisition: the performance index data of the computing terminal is collected through the monitoring client, and the performance index data is reported to the data compression tool scheduler at a fixed time, the performance index data comprises resource state data and network environment data; Matching analysis: based on the performance index data, the data compression tool of the computing terminal is scheduled through the data compression tool scheduler; Resource monitoring: the resource state data is monitored in real time, and when the resource condition deteriorates, the data resource scheduling tool is triggered to perform re-scheduling; Wherein, based on the performance index data, the data compression tool of the computing terminal is scheduled through the data compression tool scheduler, comprising the following steps: The data compression tool library of the computing terminal is obtained through the data compression tool scheduler; A performance baseline is established for each data compression tool, and the performance baseline comprises the performance under ideal conditions, the performance under general conditions, and the performance under resource limited conditions; Based on the performance index data of the previous time sequence, the performance index data of the next time sequence is predicted through the data compression tool scheduler to obtain a performance index data prediction value; Based on the performance index data prediction value and the performance baseline of the data compression tool, a matching degree score is calculated for each data compression tool in the data compression tool library of the computing terminal according to the matching degree of the resource consumption of the data compression tool and the current resource availability; The matching degree scores of different data compression tools are compared, and the data compression tool most suitable for the condition of the computing terminal is selected and scheduled from the data compression tool library of the computing terminal according to the matching degree score.

2. The computing power network-based data compression tool dynamic scheduling method according to claim 1, characterized in that, Based on the performance index data of the previous time sequence, the performance index data of the next time sequence is predicted by using a simple exponential smoothing time sequence prediction method.

3. The hashpower network-based data compression tool dynamic scheduling method according to any one of claims 1-2, characterized in that, The resource state data comprises CPU usage and memory usage, and the network environment data comprises uplink and downlink rates, packet loss rate and jitter.

4. A computing power network-based data compression tool dynamic scheduling system, characterized in that, The application is applied to a computing terminal cluster, multiple computing terminals with computing resources and network connection functions are combined into a cooperative computing terminal cluster, the computing terminal is deployed with a monitoring client, and the computing terminal cluster is deployed with a data compression tool scheduler, and the method comprises the following steps: The monitoring client is used for collecting the performance index data of the computing terminal, and reporting the performance index data to the data compression tool scheduler at a fixed time, the performance index data comprises resource state data and network environment data; The data compression tool scheduler is used for scheduling the data compression tool of the computing terminal based on the performance index data; The resource monitoring module is used for monitoring the resource state data in real time, and triggering the data resource scheduling tool to perform re-scheduling when the resource condition deteriorates; Wherein, the data compression tool scheduler is used for scheduling the data compression tool of the computing terminal as follows: The data compression tool library of the computing terminal is obtained; A performance baseline is established for each data compression tool, and the performance baseline comprises the performance under ideal conditions, the performance under general conditions, and the performance under resource limited conditions; Based on the performance index data of the last time sequence, the performance index data of the next time sequence is predicted to obtain a performance index data prediction value; Based on the performance index data prediction value and the performance baseline of the data compression tool, a matching degree score is calculated for each data compression tool in the data compression tool library of the computing terminal according to the matching degree of the resource consumption of the data compression tool and the current resource availability; The matching degree scores of different data compression tools are compared, and the data compression tool most suitable for the condition of the computing terminal is selected and scheduled from the data compression tool library of the computing terminal according to the matching degree score.

5. The hashpower network-based data compression tool dynamic scheduling system of claim 4, wherein, Based on the performance index data of the last time sequence, the data compression tool scheduler is used to predict the performance index data of the next time sequence by using a simple exponential smoothing time series prediction method.

6. The hashpower network-based data compression tool dynamic scheduling system of claim 4, wherein, The resource state data includes CPU usage and memory usage, and the network environment data includes uplink and downlink rates, packet loss rates, and jitter.

Images