A CPU computing power perception-based dynamic storage scheduling method and system

Abstract

The application provides a CPU computing power perception-based dynamic storage scheduling method and system, and belongs to the technical field of dynamic storage scheduling. The method comprises the following steps: collecting full-dimension running parameters of a storage scheduling system, dividing resource domains of a storage resource pool, and collecting multi-dimension coupling state data of each resource domain; identifying a storage scheduling abnormal scenario, combining a load stage and a hardware loss state to construct scheduling coordination risk quantitative data; generating an initial scheduling strategy based on scheduling control requirements, marking a parameter set to be adjusted according to risk data, and iteratively correcting to obtain an optimized scheduling strategy; collecting scheduling balance degree indexes, evaluating regulation and control nodes, identifying failed nodes, and generating a deviation correction adjustment strategy. The application unifies the scheduling of collaborative perception computing and storage resources, improves resource adaptability and utilization efficiency, predicts risks through multi-dimension correlation analysis, realizes active control, combines risk quantification to iteratively optimize scheduling parameters, and realizes scheduling optimization under multiple constraints.

Description

Technical Field

[0001] This invention belongs to the field of dynamic storage scheduling technology, specifically a dynamic storage scheduling method and system based on CPU computing power awareness. Background Technology

[0002] With the continuous expansion of data storage scale and the increasing complexity of business access scenarios, storage scheduling technology has become a core supporting technology for the efficient and stable operation of large-scale data systems. Its scheduling accuracy and response capability directly determine the overall system's resource utilization level and business service quality, playing a crucial role in various business scenarios with high concurrency and large data volumes.

[0003] Current mainstream storage scheduling mechanisms mostly revolve around storage-side resource allocation, exhibiting weak adaptability to the operational status of computing-side resources. This makes it difficult to dynamically match computing power with data access demands, easily leading to performance bottlenecks caused by resource supply and demand mismatches at both ends. Existing resource allocation often employs a fixed partitioning model, failing to flexibly adjust resource allocation boundaries based on real-time system operating status. In scenarios with significant business load fluctuations, local resource overload and global resource idleness often coexist, lowering overall resource utilization efficiency. Existing anomaly detection relies heavily on threshold triggers for single indicators, lacking correlation analysis of multiple operational states. This makes it difficult to identify potential performance degradation trends in advance, often resulting in reactive measures only after anomalies occur. Furthermore, existing scheduling strategies lack continuous effect verification and dynamic correction mechanisms after implementation. When some control actions fail, it is difficult to locate and adjust them in a timely manner, causing the scheduling effect to gradually decay as the system operates, making it difficult to maintain a stable optimal operating state in the long term. Summary of the Invention

[0004] This invention provides a dynamic storage scheduling method and system based on CPU computing power awareness, in order to overcome the deficiencies in the prior art.

[0005] On one hand, the present invention provides a dynamic memory scheduling method based on CPU computing power awareness, comprising: Collect all dimensions of the storage scheduling system's operating parameters, divide the storage resource pool into resource domains, and collect multi-dimensional coupling status data of each resource domain in real time.

[0006] Based on multi-dimensional coupled state data, different storage scheduling anomaly scenarios are identified. Combined with the current stage of business load and hardware wear status, scheduling coordination risk quantification data under different storage scheduling anomaly scenarios is constructed.

[0007] An initial dynamic storage scheduling strategy is generated based on preset scheduling control requirements. The set of parameters to be adjusted is marked in the initial dynamic storage scheduling strategy according to the scheduling coordination risk quantification data. The optimized scheduling strategy is obtained by performing parameter iterative correction.

[0008] The scheduling balance index is collected in real time from the coupled status data of each resource domain after scheduling execution. Each control node in the optimized scheduling strategy is comprehensively evaluated. Control nodes that fail to meet the evaluation are defined as control failure nodes, and a correction and adjustment strategy is generated for the control failure nodes.

[0009] According to the CPU computing power-aware dynamic storage scheduling method provided by the present invention, the process of partitioning the storage resource pool into resource domains includes: The server NUMA node topology, CPU core group distribution, storage medium array layout parameters, and business data deployment granularity are collected as full-dimensional operating parameters.

[0010] Based on the computing power-storage affinity requirements of business tasks and the target scheduling response latency, NUMA nodes are used as the basic unit to meet the condition of uniform computing power supply. The storage resource pool is regularly divided according to the distribution location of the storage medium array to determine the number of resource domains.

[0011] The storage access boundary of each resource domain is defined based on the number of interconnection hops between the data block and the CPU core group.

[0012] According to the CPU computing power-aware dynamic storage scheduling method provided by the present invention, the process of collecting multi-dimensional coupled state data includes: Collect user-mode utilization and kernel-mode utilization of CPU core groups, calculate total computing power utilization and inter-domain computing power difference, and form CPU computing power margin data.

[0013] Random read IOPS, sequential write IOPS, and average IO response latency are collected in each resource domain. At the same time, the IO queue depth and data block size are recorded and normalized according to the standard data block size to obtain the storage IO throughput characteristics.

[0014] The access frequency and recent access time interval of data blocks in each resource domain are collected according to a preset frequency. The data heat value is obtained based on the heat calculation model. The heat distribution concentration and hot spot shift time period are analyzed to form a data access heat hierarchy.

[0015] The cache hit rate is calculated by comparing the number of cache hits in each resource domain with the total number of accesses within the statistical period, and the cache hit efficiency is obtained based on the cache hit calculation formula.

[0016] The real-time data transmission rate of different storage tiers is measured on the interconnect links of each resource domain. The bandwidth utilization rate of the corresponding tier is calculated based on the rated bandwidth of the link. The bandwidth difference between different tiers is recorded to obtain the interconnect link bandwidth utilization data.

[0017] Time-series alignment is performed on CPU computing power margin, storage I / O throughput characteristics, data access popularity level, cache hit efficiency, and interconnect link bandwidth utilization data within the same resource domain. Correlation characteristics between different parameters are marked to obtain multi-dimensional coupling state data.

[0018] According to the CPU computing power-aware dynamic memory scheduling method provided by the present invention, the process of identifying different memory scheduling anomaly scenarios includes: The coupling state data of each resource domain is standardized and aligned in spatiotemporal dimension according to the resource domain number and time slice to obtain the coupling state processing data.

[0019] From the coupled state processing data, extract single-parameter features and multi-parameter coupling features related to storage scheduling anomalies as scheduling anomaly features.

[0020] Based on the business performance requirements of dynamic storage scheduling, three levels of thresholds are set for scheduling anomaly characteristics.

[0021] An abnormal scenario matching rule base is established based on the three-level threshold and scheduling anomaly characteristics. The coupling state processing data of each resource domain is traversed, and the corresponding storage scheduling anomaly scenario is determined by combining the matching rule base.

[0022] According to the CPU computing power-aware dynamic storage scheduling method provided by the present invention, the process of constructing scheduling collaborative risk quantification data includes: The absolute deviation of performance indicators from the target value for different storage scheduling anomaly scenarios is converted into relative deviation rates. Based on the sensitivity of different load stages to scheduling anomaly scenarios, a basic weight for the load stage is set, and the load stage sensitivity factor is calculated.

[0023] The CPU computing performance degradation rate, disk read / write performance degradation rate, monitoring probe accuracy deviation rate, cache efficiency degradation rate, and hardware cumulative runtime are normalized and weighted to obtain the hardware loss coefficient.

[0024] For storage scheduling anomaly scenarios, risk factors for anomaly scenarios are classified from three dimensions: the severity of the anomaly type, the scope of its impact, and the duration of its duration.

[0025] The loss amplification factor is calculated based on the hardware loss coefficient.

[0026] By using a weighted summation method, the risk index for collaborative scheduling is calculated by coupling the risk factors of abnormal scenarios, the sensitivity factors of load phases, and the loss amplification factor.

[0027] The collaborative scheduling risk index is divided into multiple risk levels based on a preset numerical range, and the risk characteristics and control requirements corresponding to each risk level are determined to obtain quantitative data on collaborative scheduling risks.

[0028] According to the CPU computing power-aware dynamic memory scheduling method provided by the present invention, the process of generating an initial dynamic memory scheduling policy includes: The system throughput target, IO response latency constraint, and system energy consumption threshold are taken as scheduling control requirements, broken down into target values ​​of specific scheduling parameters, and weight coefficients of each target are set according to business priority.

[0029] Based on historical scheduling data, computing power allocation parameters, IO scheduling parameters, and data migration parameters are selected as key scheduling parameters, and a target correlation model between key scheduling parameters and scheduling target values ​​is established.

[0030] Based on the business load phase, parameter threshold ranges for each phase are set in conjunction with the target values ​​of scheduling parameters, and a preliminary scheduling strategy is generated according to the linkage rules between different parameters.

[0031] The initial scheduling strategy is validated from three dimensions: system throughput, IO response latency, and system energy consumption. Multi-objective optimization is performed according to the target weight coefficients to obtain the initial dynamic storage scheduling strategy.

[0032] According to the CPU computing power-aware dynamic memory scheduling method provided by the present invention, the process of obtaining the optimized scheduling strategy includes: We analyzed the impact parameters corresponding to the scheduling coordination risk quantification data of different storage scheduling anomaly scenarios, determined the risk mitigation logic of parameter adjustment, and constructed a risk-parameter association rule base.

[0033] For each storage scheduling anomaly scenario, calculate the sensitivity coefficient of each influencing parameter.

[0034] Based on the risk-parameter association rule base, the parameter categories, adjustment ranges, and adjustment priorities that need to be adjusted are marked in the initial dynamic storage scheduling policy to obtain the set of parameters to be adjusted.

[0035] Based on the mitigation logic, determine the direction of parameter adjustment, calculate the adjustment range by combining the risk level and sensitivity coefficient, design the sequence of parameter adjustments for multi-parameter linkage adjustment scenarios, and generate parameter adjustment schemes.

[0036] Substitute the parameter adjustment scheme into the initial dynamic storage scheduling strategy, and determine whether it comprehensively meets the scheduling control requirements. If it does, generate an optimized scheduling strategy; otherwise, correct the adjustment magnitude and direction and re-verify.

[0037] According to the CPU computing power-aware dynamic memory scheduling method provided by the present invention, the process of obtaining the scheduling balance index includes: After the scheduling is executed, the coupled status data of each resource domain is collected. Based on the status change characteristics of each parameter, a differentiated collection frequency is set, and the data is normalized to obtain the scheduling processing data.

[0038] The mean, standard deviation, and coefficient of variation of the same parameter are calculated for each resource domain of the entire system. For a single resource domain, the fluctuation amplitude and frequency of the parameter within a continuous acquisition period are calculated to obtain the single-index equilibrium characteristics.

[0039] Based on the single-index balance characteristics, and combining the two dimensions of inter-domain consistency and temporal stability, corresponding weights are assigned to calculate the single-parameter scheduling balance.

[0040] By combining the weights of the impact of different parameters on storage scheduling performance, the corresponding single-parameter scheduling balance is weighted and summed to obtain the scheduling balance index.

[0041] According to the CPU computing power-aware dynamic storage scheduling method provided by the present invention, the process of generating a correction and adjustment strategy includes: The optimized scheduling strategy is categorized and sorted by load stage, control resource domain, and parameter type. All control nodes are identified, and the parameter settings, control range, and control objects of each control node are determined to obtain node characteristic data.

[0042] By using historical control data, the weight of the deviation impact of each control node on the scheduling balance index is calculated.

[0043] For each control node, based on its node characteristic data and deviation impact weight, a comprehensive evaluation is conducted according to the preset scheduling balance index threshold and the preset node control deviation threshold to obtain the node evaluation result.

[0044] Node failure degree is calculated based on node evaluation results, node failure level is divided according to preset value range, and nodes that reach the preset failure level are selected as control failure nodes.

[0045] For each control failure node, analyze the root cause of the control failure, and generate corresponding corrective adjustment strategies based on the problem type of the root cause and the failure level of the node.

[0046] On the other hand, the present invention also provides a CPU-power-aware dynamic memory scheduling system, comprising: The multi-domain coupling monitoring module is used to collect all-dimensional operating parameters of the storage scheduling system, divide the storage resource pool into resource domains, and collect data on CPU computing power margin, storage IO throughput characteristics, data access popularity level, cache hit efficiency and interconnect link bandwidth utilization of each resource domain in real time as multi-dimensional coupling status data.

[0047] The scheduling risk quantification and assessment module is used to identify different storage scheduling anomaly scenarios based on multi-dimensional coupled state data. It combines the current stage of business load and hardware wear status to construct scheduling collaborative risk quantification data under different storage scheduling anomaly scenarios.

[0048] The dynamic scheduling strategy generation module is used to generate an initial dynamic storage scheduling strategy based on preset scheduling control requirements, mark the set of parameters to be adjusted in the initial dynamic storage scheduling strategy according to the scheduling coordination risk quantification data, and perform parameter iterative correction to obtain an optimized scheduling strategy.

[0049] The control node feedback correction module is used to collect the scheduling balance index of the coupling status data of each resource domain after scheduling execution in real time, to comprehensively evaluate each control node in the optimized scheduling strategy, to define the control node that fails to meet the evaluation as the control failure node, and to generate a correction adjustment strategy pointing to the control failure node.

[0050] This invention provides a dynamic storage scheduling method and system based on CPU computing power awareness. By comprehensively collecting all dimensions of system operating parameters and performing regular resource allocation, it realizes the collaborative awareness and unified scheduling of computing resources and storage resources. It can match the corresponding data access load according to the computing supply capacity, improve the adaptability and overall utilization efficiency of the two types of resources, and alleviate the performance loss caused by uneven resource distribution.

[0051] By analyzing the correlation between various operating states, different abnormal situations can be identified. By combining the business load stage and hardware wear status, scheduling risks can be quantified, enabling early prediction and hierarchical control of risks. This transforms passive anomaly handling into proactive risk avoidance, reduces the impact of scheduling conflicts and performance degradation on business operations, and ensures the continuity of business operations.

[0052] An initial strategy is generated based on scheduling and control requirements, and parameters are iteratively corrected by combining risk quantification results. This enables the scheduling strategy to adapt to the current system state and risk level, achieving optimal scheduling performance under multiple constraints of performance, latency, and energy consumption, and ensuring long-term stable service quality of the system. Attached Figure Description

[0053] The invention will now be further described with reference to the accompanying drawings.

[0054] Figure 1 This is a flowchart illustrating a dynamic storage scheduling method based on CPU computing power awareness in this invention. Figure 2 This is a flowchart illustrating the process of identifying different storage scheduling anomaly scenarios in this invention; Figure 3 This is a schematic diagram of the process for constructing scheduling and collaborative risk quantification data in this invention; Figure 4 This is a flowchart illustrating the process of obtaining the optimized scheduling strategy in this invention; Figure 5 This is a schematic diagram of the structure of a dynamic storage scheduling system based on CPU computing power awareness in this invention. Detailed Implementation

[0055] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0056] like Figures 1 to 5 As shown in the figure, the embodiment of the present invention provides a dynamic memory scheduling method and system based on CPU computing power awareness. The execution subject can be a dynamic memory scheduling method based on CPU computing power awareness, and the method includes: Collect all dimensions of the storage scheduling system's operating parameters and divide the storage resource pool into resource domains.

[0057] The server NUMA node topology, CPU core group distribution, storage medium array layout parameters, and business data deployment granularity are collected as full-dimensional operating parameters.

[0058] The NUMA node topology represents the physical connection relationship of each node in a non-uniform memory access architecture, including the type of interconnection link between nodes, link bandwidth, and access latency parameters; the CPU core distribution location represents the mapping relationship between physical cores and hyper-threaded logical cores within each NUMA node and the core numbering distribution; the storage media array layout parameters include the slot distribution of solid-state drive arrays and hard disk arrays, controller affiliation, bus interface type, and single disk capacity; the service data deployment granularity represents the smallest allocation unit of service data on the storage medium, and the data block size can be set to 4KB, 8KB, or 16KB. In this embodiment, 4KB is used as the standard data block size.

[0059] Based on the computing power-storage affinity requirements of business tasks and the target scheduling response latency, NUMA nodes are used as the basic unit to meet the condition of uniform computing power supply. The storage resource pool is regularly divided according to the distribution location of the storage medium array to determine the number of resource domains.

[0060] The computing power-storage affinity requirement means that computing tasks and accessed data should be deployed on the same NUMA node as much as possible to avoid performance loss caused by cross-node interconnection links. The quantitative indicator of affinity requirement is that the proportion of cross-node access should not exceed a preset proportion of the total access volume, such as not exceeding 15%.

[0061] The target scheduling response delay represents the maximum allowable time from the issuance of the scheduling instruction to the completion of the scheduling action. In this embodiment, it is set to 500 microseconds.

[0062] Using NUMA nodes as the basic unit, the condition of uniform computing power supply is met, and the difference in the number of CPU logical cores allocated in each resource domain does not exceed a preset threshold, for example, the difference in the number of cores in each resource domain does not exceed 2.

[0063] The storage resource pool is organized and divided according to the distribution of the storage media array. Storage media that are physically directly connected to the same NUMA node are assigned to the same resource domain to ensure that the bus path for storage access within the domain is the shortest.

[0064] The number of resource domains is determined according to the following principles: when the total capacity of the storage media mounted on a single NUMA node exceeds the single domain management limit, the node is further subdivided into multiple subdomains according to the storage controller affiliation; when multiple NUMA nodes share the same storage controller, the shared storage media is independently divided into cross-node shared domains.

[0065] In a server comprising 4 NUMA nodes, each node mounting 8 NVMe disks and 12 SATA disks, the resources can be divided into 4 local resource domains and 1 cross-node shared domain, for a total of 5 resource domains.

[0066] Based on the interconnection hops between data blocks and CPU cores, the storage access boundary of each resource domain is defined to ensure the shortest data access path within the domain.

[0067] Interconnect hop count refers to the number of NUMA nodes that data must traverse from the storage medium to the target CPU core group. A smaller hop count indicates a shorter access path and lower access latency.

[0068] When defining storage access boundaries, the number of interconnect hops not exceeding a preset threshold is used as the criterion for determining domain affiliation. The hop count threshold is set to 0, meaning that only storage media directly connected to this NUMA node are included in this resource domain.

[0069] For storage media in a cross-node shared domain, priority is assigned to the node to which the CPU core group with the highest access frequency belongs, and the number of hops to other nodes is recorded.

[0070] To ensure the shortest data access path within the domain, boundary delineation is used to ensure that the average number of hops for CPU cores accessing storage media within the domain is less than the average number of hops for accessing storage media outside the domain.

[0071] Based on the number of resource domains and storage access boundaries, performance acquisition probes, IO monitoring units, and link traffic statistics modules are deployed in each resource domain, and global clock alignment and performance baseline calibration of each monitoring unit are completed.

[0072] The performance acquisition probe is a lightweight monitoring program that runs in the CPU kernel mode. It reads the performance counters in specific registers of the CPU model at fixed time intervals to collect hardware performance events such as the number of user-mode instruction cycles, the number of kernel-mode instruction cycles, and the number of cache misses. The acquisition interval is set to 1 millisecond.

[0073] The IO monitoring unit is deployed in the storage controller driver layer, intercepting and recording the initiation time, completion time, request type, data block address, and data length of each IO request, and supporting statistical aggregation by resource domain.

[0074] The link traffic statistics module is deployed at the PCIe root complex and NUMA interconnect controller to count the number of bytes transmitted, data packets, and link utilization in real time for each link. The statistics period is set to 10 milliseconds.

[0075] Global clock alignment refers to unifying the timestamps of all monitoring units to the system's global clock reference, and controlling the time deviation between units within a preset range through a cross-core synchronization mechanism of CPU timestamp counters.

[0076] Performance baseline calibration involves continuously collecting performance data for a preset duration under system no-load conditions, calculating the baseline values ​​and normal fluctuation ranges of each parameter, and using this data as a reference baseline for anomaly detection.

[0077] The performance baseline calibration process is as follows: stop all business loads and confirm that the system is in an idle state, start continuous acquisition of all parameters, calculate the mean and standard deviation of each parameter every 5 minutes during the acquisition process, and determine that the calibration is completed when the mean fluctuation of the three consecutive calculation results is less than 1% and the standard deviation fluctuation is less than 5%. If the calibration time reaches 60 minutes and still has not converged, take the mean of the last 10 minutes as the baseline value and mark the calibration status as approximately converged.

[0078] Real-time data collection of CPU computing power margin, storage I / O throughput characteristics, data access popularity level, cache hit efficiency, and interconnection link bandwidth utilization in each resource domain serves as multi-dimensional coupled status data.

[0079] Based on the multi-source monitoring units deployed in each resource domain, the user-mode utilization and kernel-mode utilization of CPU core groups are collected, the total computing power utilization and the computing power difference between domains are calculated, and CPU computing power margin data is formed.

[0080] User-mode utilization represents the percentage of time a CPU core spends executing user-space application instructions within a statistical period, while kernel-mode utilization represents the percentage of time a CPU core spends executing operating system kernel instructions, including system calls, interrupt handling, and kernel thread execution time.

[0081] The total computing power utilization rate is the sum of the user-mode utilization rate and the kernel-mode utilization rate. The idle state ratio is obtained by subtracting the total computing power utilization rate from 100%, and does not need to be collected separately.

[0082] CPU computing power margin represents the proportion of currently available computing power to total computing power. It is calculated as the product of the idle state percentage and the computing power reservation coefficient. The computing power reservation coefficient is used to guarantee the minimum computing resources required for basic system operation, and its calculation formula is as follows: In the formula, For CPU computing power margin, This represents the total computing power utilization rate. A coefficient is reserved for computing power.

[0083] Inter-domain computing power disparity represents the degree of dispersion among the total computing power utilization rates of each resource domain. It is expressed as the ratio of the standard deviation of each domain's utilization rate to the average utilization rate of the entire system. The calculation formula is as follows: In the formula, To represent the difference in computing power between domains, The total number of resource domains, For the first Total computing power utilization of each resource domain This is the arithmetic mean of the total computing power utilization rate of all resource domains in the entire system.

[0084] CPU computing power margin data includes user-mode utilization, kernel-mode utilization, total computing power utilization, computing power margin, and inter-domain computing power differences for each resource domain, and is stored in time-series format.

[0085] Random read IOPS, sequential write IOPS, and average IO response latency are collected in each resource domain. At the same time, the IO queue depth and data block size are recorded and normalized according to the standard data block size to obtain the storage IO throughput characteristics.

[0086] Random read IOPS represents the number of random read operations completed per unit of time. A random operation is defined as an operation in which the offset between the access address and the previous access address exceeds a preset threshold. For example, if the offset exceeds 128KB, it is considered a random access.

[0087] Sequential write IOPS represents the number of sequential write operations completed per unit of time. Sequential operations are defined as operations that access addresses that are continuously incremented and whose offsets do not exceed a preset threshold.

[0088] Average IO response latency represents the average time interval from when an IO request is submitted to the storage controller to when a request completion notification is returned, including queue waiting time, transmission time, and storage media processing time.

[0089] IO queue depth represents the number of IO requests that are active simultaneously in the storage controller, reflecting the concurrent processing pressure of the storage system.

[0090] The data block size represents the length of data operated by a single IO request. The data block size may vary in different business scenarios. For example, 8KB is the main size in database transaction scenarios, while 128KB is the main size in large file transfer scenarios.

[0091] The IOPS of different data block sizes are normalized to the standard data block size, and the IOPS of different data block sizes are converted to the equivalent IOPS under the standard data block size. The correction formula is as follows: In the formula, For the normalized equivalent IOPS, For the original IOPS, The actual requested data block size. The standard data block size is set to 4KB in this embodiment.

[0092] Storage I / O throughput characteristics include normalized random read IOPS, normalized sequential write IOPS, average I / O response latency, I / O queue depth, and data block size distribution for each resource domain.

[0093] The access frequency and recent access time interval of data blocks in each resource domain are collected according to a preset frequency. The data heat value is obtained based on the heat calculation model. The heat distribution concentration and hot spot shift time period are analyzed to form a data access heat hierarchy.

[0094] The preset frequency represents the length of the time window for data block access statistics. In this embodiment, it is set to 5 seconds, that is, the access status of each data block within the time window is counted every 5 seconds.

[0095] Access frequency represents the total number of times a single data block is read or written within a statistical time window.

[0096] The most recent access interval represents the time difference between the current moment and the last time the data block was accessed, reflecting the temporal locality of the data.

[0097] The data popularity value is obtained based on the popularity calculation model. The popularity calculation comprehensively considers two dimensions: access frequency and temporal locality. The higher the access frequency and the shorter the time interval between recent accesses, the higher the data popularity value. The calculation formula is as follows: In the formula, For the first The heat value of each data block This represents the access frequency of the data block per unit of time. This is the time interval between the most recent accesses to this data block. , Calculate the weighting coefficient for popularity. This represents the maximum data popularity within the corresponding resource domain, used for normalization.

[0098] The weighting coefficients are adjusted according to the business type; for read-intensive businesses, they can be set. , For write-intensive tasks, you can set... , .

[0099] The concentration of popularity distribution indicates the degree of concentration of access volume among data blocks, and is expressed as the proportion of access volume carried by the top 10% of data blocks in terms of popularity value to the total access volume.

[0100] Hotspot shift period refers to the time period during which the set of high-popularity data blocks changes over time. It is identified by comparing the overlap of the top 10% of data blocks in popularity values ​​within adjacent statistical windows. When the overlap is lower than a preset threshold, it is determined that a hotspot shift has occurred.

[0101] Data access popularity levels are divided into high-hot, medium-hot, and low-hot layers according to the range of popularity values. For example, a popularity value greater than 0.7 is a high-hot layer, 0.3 to 0.7 is a medium-hot layer, and less than 0.3 is a low-hot layer. Different cache priorities and storage media placement strategies are adopted for data blocks at each level.

[0102] The cache hit rate is calculated by comparing the number of cache hits in each resource domain with the total number of accesses within the statistical period, and the cache hit efficiency is obtained based on the cache hit calculation formula.

[0103] The cache includes CPU caches at various levels and memory page caches. The CPU cache is divided into L1 instruction cache, L1 data cache, L2 cache and L3 cache, while the memory page cache is located in main memory and is used to cache disk data blocks.

[0104] Cache hit count indicates the number of times a data access request finds the required data in the cache without needing to access the underlying storage medium.

[0105] The total number of accesses within the statistical period represents the total number of data access requests received by the cache, including both hits and misses.

[0106] Cache hit rate reflects the speedup effect of caching on storage access. A higher hit rate indicates a lower proportion of requests that need to access slow storage media, resulting in better overall system I / O performance. The calculation formula is: In the formula, For cache hit rate, To count the number of cache hits within a statistical period, This represents the total number of data accesses within the statistical period.

[0107] Cache hit efficiency data is statistically analyzed separately for each cache level, including L1 cache hit rate, L2 cache hit rate, L3 cache hit rate and memory page cache hit rate. The hit rate for each level can be set to 95% to 99% for L1 cache, 90% to 95% for L2 cache, 80% to 90% for L3 cache, and 60% to 85% for memory page cache.

[0108] The real-time data transmission rate of different storage tiers is measured on the interconnect links of each resource domain. The bandwidth utilization rate of the corresponding tier is calculated based on the rated bandwidth of the link. The bandwidth difference between different tiers is recorded to obtain the interconnect link bandwidth utilization data.

[0109] Interconnect links include PCIe bus links and NUMA node interconnect links. PCIe bus links are used for data transfer between the CPU and the storage medium, while NUMA interconnect links are used for data transfer between node memory access and node storage access.

[0110] Different storage tiers are represented by the NVMe solid-state drive tier, the SATA hard disk drive tier, and the memory tier, with each tier connected to the CPU through different bus interfaces.

[0111] Real-time data transmission rate represents the number of valid data bytes passing through the link per unit time.

[0112] Bandwidth utilization is the ratio of real-time data transmission rate to the link's rated bandwidth, calculated using the following formula: In the formula, For the first Bandwidth utilization of interconnect links, This represents the real-time data transmission rate of the link. This is the rated bandwidth of the link.

[0113] The bandwidth difference between different layers is represented by the difference in bandwidth utilization of each layer. For example, when the bandwidth utilization of the NVMe layer is 70% and the bandwidth utilization of the SATA layer is 20%, the bandwidth difference between the two layers is 50 percentage points, reflecting the uneven distribution of storage load between different media layers.

[0114] Interconnect link bandwidth utilization data includes real-time utilization, peak utilization, and average utilization for each link within each resource domain.

[0115] Based on the system global clock, time-series alignment is performed on CPU computing power margin, storage I / O throughput characteristics, data access heat level, cache hit efficiency and interconnect link bandwidth utilization data within the same resource domain, and the correlation characteristics between different parameters are marked to obtain multi-dimensional coupling state data.

[0116] Timing alignment means unifying parameter data from different sampling periods onto the same time slice grid. Linear interpolation or nearest neighbor sampling methods are used to downsample high-frequency data or upsample low-frequency data so that all parameters have corresponding data values ​​on the same time slice.

[0117] The correlation characteristics between different parameters are marked, indicating that positive correlation, negative correlation or non-linear coupling relationship between parameters can be identified through correlation analysis. For example, there may be a positive correlation between CPU computing power utilization and IO response latency, a positive correlation between cache hit rate and IO throughput, and a positive correlation between link bandwidth utilization and IO queue depth.

[0118] The correlation features are quantified using the Pearson correlation coefficient. A correlation coefficient with an absolute value greater than a preset threshold of 0.7 is considered a strong correlation, 0.3 to 0.7 is considered a moderate correlation, and less than 0.3 is considered a weak correlation.

[0119] Multidimensional coupled state data is indexed by resource domain number and time slice, and includes parameter values ​​for each dimension and the correlation characteristics between parameters.

[0120] The data structure of multi-dimensional coupled state data is defined as follows: each record includes a resource domain number, time slice start timestamp, CPU computing power margin value, storage IO throughput characteristic structure, data access heat hierarchy structure, cache hit efficiency structure, interconnect link bandwidth utilization structure, and parameter association characteristic matrix. The data is written to the memory circular buffer in columnar storage format.

[0121] Identify different storage scheduling anomaly scenarios based on multi-dimensional coupled state data.

[0122] The coupling state data of each resource domain is standardized and aligned in spatiotemporal dimension according to the resource domain number and time slice to obtain the coupling state processing data.

[0123] Parameters with different dimensions and ranges are converted into dimensionless values ​​of a uniform scale using the Z-score standardization method. This method centers on the parameter mean and uses the standard deviation as the unit of transformation. The transformed data has a mean of 0 and a standard deviation of 1. The calculation formula is as follows: In the formula, For the first Each resource domain in time slice Standardized parameter values, These are the original parameter values. This is the average value of this parameter across all resource domains and time slices in the entire system. This represents the standard deviation of the parameter.

[0124] Construct a parameter matrix with resource domains as rows and time slices as columns, ensuring that each matrix element corresponds to a unique combination of resource domain and time slice. Missing data is filled in using forward padding or linear interpolation.

[0125] The coupled state processing data is a standardized multidimensional spatiotemporal matrix, and each matrix element includes the standardized values ​​of all parameters of the resource domain in that time slice.

[0126] From the coupled state processing data, extract single-parameter features and multi-parameter coupling features related to storage scheduling anomalies as scheduling anomaly features.

[0127] The single-parameter features include the instantaneous value of the parameter, the mean of the sliding window, the standard deviation of the sliding window, the rate of change, and the extreme values. The sliding window length is set to 10 time slices, or 100 milliseconds.

[0128] The rate of change is the ratio of the difference between parameter values ​​in adjacent time slices to the length of the time slice, reflecting the degree of drastic change in the parameter.

[0129] Multi-parameter coupling characteristics include changes in the correlation coefficient between parameters, the magnitude of the coupling strength deviating from the baseline value, and coupling direction reversal events.

[0130] The change in the correlation coefficient represents the difference between the correlation coefficient between parameters within the current sliding window and the baseline correlation coefficient.

[0131] The magnitude of the coupling strength deviation from the baseline value is used to identify abnormal changes in the relationship between parameters. A coupling direction reversal event indicates that the parameters change from positive to negative or vice versa, indicating a fundamental change in the system state.

[0132] The scheduling anomaly features are a set of single-parameter features and multi-parameter coupled features, with each feature associated with a corresponding resource domain number and time slice.

[0133] Based on the business performance requirements of dynamic storage scheduling, three levels of thresholds are set for the extracted scheduling anomaly features: early warning, minor anomaly, and severe anomaly.

[0134] Business performance requirements include the lower limit of system throughput, the upper limit of I / O response latency, and system availability requirements. The three threshold levels are set based on performance baseline calibration data and business tolerance. The warning threshold corresponds to the state where the performance index is close to the lower or upper limit of business requirements. The mild anomaly threshold corresponds to the state where the performance index exceeds the business requirements but has not yet caused business interruption. The severe anomaly threshold corresponds to the state where the performance index seriously exceeds the business requirements and may cause business interruption.

[0135] For average IO response latency, the warning threshold can be set to 400 microseconds, the threshold for minor anomalies can be set to 800 microseconds, and the threshold for severe anomalies can be set to 2 milliseconds.

[0136] For total CPU utilization, the warning threshold can be set to 80%, the threshold for minor anomalies to 90%, and the threshold for severe anomalies to 97%.

[0137] For system throughput, the warning threshold can be set to 90% of the throughput target, i.e., 72,000 IOPS; the threshold for minor anomalies can be set to 80% of the throughput target, i.e., 64,000 IOPS; and the threshold for severe anomalies can be set to 60% of the throughput target, i.e., 48,000 IOPS.

[0138] The three threshold levels are configured separately for parameter type and resource domain. Different threshold standards can be used for resource domains with different service priorities.

[0139] An abnormal scenario matching rule base is established based on the three-level threshold and scheduling anomaly characteristics. The coupling state processing data of each resource domain is traversed, and the corresponding storage scheduling anomaly scenario is determined by combining the matching rule base.

[0140] The abnormal scenario matching rule base consists of multiple rules. Each rule includes a combination of triggering conditions and a corresponding abnormal scenario type. The combination of triggering conditions is described by a combination of logical AND, logical OR, and logical NOT expressions.

[0141] Storage scheduling anomalies include computing power overload, I / O bottleneck, cache failure, link congestion, hotspot shift, and hardware anomaly.

[0142] In this embodiment, the triggering conditions for the computing power overload scenario are: the total CPU computing power utilization rate exceeds the mild anomaly threshold and the inter-domain computing power difference exceeds 0.2; the triggering conditions for the IO bottleneck scenario are: the average IO response latency exceeds the mild anomaly threshold and the IO queue depth exceeds the preset threshold; the triggering conditions for the cache failure scenario are: the cache hit rate is lower than the warning threshold and the heat distribution concentration exceeds 0.8; the triggering conditions for the link congestion scenario are: the bandwidth utilization rate of a certain interconnect link exceeds the severe anomaly threshold and the duration exceeds the preset duration; the triggering conditions for the hot spot offset scenario are: the overlap of high-hot data blocks in adjacent statistical windows is less than 60% and the data migration bandwidth utilization rate exceeds 50%; the triggering conditions for the hardware anomaly scenario are: the IO error rate of the storage medium in a certain resource domain exceeds the preset threshold, or the correctable error count of the CPU core exceeds the preset threshold within the statistical period, or the link traffic statistics module detects that the link retransmission rate exceeds 1%.

[0143] When processing data by traversing the coupling state of each resource domain, the rules in the rule base are matched one by one in the order of time slices. When all the triggering conditions of a certain rule are met, it is determined that the current resource domain has the corresponding abnormal scenario in the time slice, and the abnormal scenario type, occurrence time, duration and abnormal values ​​of the parameters involved are recorded.

[0144] Based on the current stage of business load and hardware wear status, construct quantitative data on scheduling coordination risks under different storage scheduling anomaly scenarios.

[0145] The absolute deviation of performance indicators from the target value for different storage scheduling anomaly scenarios is converted into relative deviation rates. Based on the sensitivity of different load stages to scheduling anomaly scenarios, a basic weight for the load stage is set, and the load stage sensitivity factor is calculated.

[0146] The absolute deviation of a performance indicator from its target value represents the absolute value of the difference between the current measured value of the performance indicator and the business target value.

[0147] The relative deviation rate is the ratio of the absolute deviation to the target value. In this embodiment, the relative deviation rate is 60%.

[0148] The business load is divided into low load, medium load, high load and peak load stages. The division of each stage is based on the total computing power utilization rate of the entire system. For example, a total computing power utilization rate of less than 30% is a low load stage, 30% to 60% is a medium load stage, 60% to 85% is a high load stage, and more than 85% is a peak load stage.

[0149] Different load phases are sensitive to scheduling anomalies to varying degrees. The same deviation from the schedule under high load and peak load phases has a greater impact on business performance. Therefore, higher base weights are set, for example, 0.3 for low load phase, 0.5 for medium load phase, 0.8 for high load phase, and 1.0 for peak load phase.

[0150] The load phase sensitivity factor is the product of the relative deviation rate and the load phase base weight, calculated using the following formula: In the formula, For load phase sensitivity factors, These are the current measured values ​​of the performance indicators. For the target value of the performance index, This represents the base weight corresponding to the current load phase.

[0151] The hardware wear coefficient is obtained by normalizing the CPU computing performance degradation rate, disk read / write performance degradation rate, memory channel performance degradation rate, cache efficiency degradation rate, and hardware cumulative runtime, and then weighting them.

[0152] CPU computing performance degradation rate represents the percentage decrease in computing performance caused by prolonged high-load operation of the CPU. It is calculated by comparing the current benchmark score with the factory benchmark score. For example, if the current score is 95% of the factory score, the degradation rate is 5%.

[0153] Disk read / write performance degradation rate indicates the percentage decrease in read / write speed due to the cumulative number of erase / write cycles of the storage medium. The performance degradation of NVMe solid-state drives is related to the number of flash memory erase / write cycles, while the performance degradation of mechanical hard drives is related to head wear and the number of bad sectors.

[0154] The memory channel performance degradation rate represents the percentage reduction in memory read / write bandwidth caused by memory cell aging and signal integrity degradation. It is calculated through memory bandwidth benchmark tests.

[0155] Cache efficiency decay rate represents the percentage decrease in effective cache capacity due to cache line conflicts, cache pollution, or memory cell aging.

[0156] The cumulative hardware runtime represents the total amount of time the device has been powered on and running since it was put into use, in hours.

[0157] Each loss index is converted into a value in the range of 0 to 1. A linear mapping method is used, with the maximum allowable loss value of each index as the normalization denominator. For example, the maximum allowable decay rate of CPU computing performance is 20%, the maximum allowable decay rate of disk read and write performance is 30%, and the maximum allowable decay rate of memory channel performance is 25%.

[0158] The hardware loss factor is the weighted sum of all normalized loss indices, and the calculation formula is as follows: In the formula, This is the hardware loss factor. Calculate the normalized value of the CPU performance degradation rate. This is the normalized value of the disk read / write performance degradation rate. This is the normalized value of the memory channel performance degradation rate. This is a normalized value for cache efficiency decay rate. The normalized value of accumulated hardware runtime. to Let be the weighting coefficients for each loss dimension, and satisfy . .

[0159] The weighting coefficients are determined using the analytic hierarchy process (AHP). A judgment matrix is ​​constructed for each loss dimension relative to the overall hardware loss target. The eigenvector corresponding to the largest eigenvalue of the judgment matrix is ​​calculated as the weight. A consistency check is performed, and the weighting result is accepted when the consistency ratio is less than 0.1.

[0160] For example, for an all-flash storage system, the analytic hierarchy process can be used to set... , , , , .

[0161] For storage scheduling anomaly scenarios, risk factors for anomaly scenarios are classified from three dimensions: the severity of the anomaly type, the scope of its impact, and the duration of its duration.

[0162] The severity level indicates the potential damage that abnormal scenarios may cause to business performance and data security. It is divided into three levels: low, medium, and high. For example, the severity level of a computing power overload scenario is medium, the severity level of an IO bottleneck scenario is high, and the severity level of a hardware anomaly scenario is high. The severity levels are 0.3 for low, 0.6 for medium, and 1.0 for high.

[0163] The scope of impact indicates the number of resource domains and business tasks affected by the abnormal scenario. The scope of impact for a single resource domain and a single business is small, for a single resource domain and a single business it is medium, and for a large scope it is large. For example, small scope is 0.3, medium scope is 0.6, and large scope is 1.0.

[0164] Duration indicates the length of time from the occurrence of an abnormal scenario to its recovery. Less than 1 second is considered short, 1 to 10 seconds is considered medium, and more than 10 seconds is considered persistent. For example, short is 0.3, medium is 0.6, and persistent is 1.0.

[0165] The risk factor for abnormal scenarios is a weighted sum of three quantified dimensions. The weighting coefficients are determined using the entropy weighting method based on the information entropy of historical abnormal data. The calculation formula is as follows: In the formula, As a risk factor for abnormal scenarios, This is a quantification of the degree of harm. To quantify the range of influence, This is a quantified value for duration. , , These are the weight coefficients for the corresponding dimensions.

[0166] The loss amplification factor is calculated based on the hardware loss coefficient.

[0167] The loss amplification factor reflects the amplification effect of hardware loss status on the risk of abnormal scenarios. The more severe the hardware loss, the higher the actual risk under the same abnormal scenario.

[0168] The loss amplification factor is calculated using a nonlinear mapping function with the hardware loss coefficient as input. When the hardware loss coefficient is low, the amplification factor is close to 1; when the hardware loss coefficient exceeds a preset threshold, the amplification factor increases rapidly. The calculation formula is as follows: In the formula, This is the loss amplification factor. This is the hardware loss factor. This is the amplification factor, used to control the amplification of risk by losses, and is set to 2.0.

[0169] When the hardware loss factor is 0.2, the loss amplification factor is 1.08; when the hardware loss factor is 0.5, the loss amplification factor is 1.5.

[0170] By using a weighted summation method, the risk index for collaborative scheduling is calculated by coupling the risk factors of abnormal scenarios, the sensitivity factors of load phases, and the loss amplification factor.

[0171] The collaborative scheduling risk index comprehensively reflects the risk level of abnormal scenarios, the sensitivity of the current load phase, and the amplification effect of hardware losses. It is a core risk quantification indicator for scheduling decisions, and its calculation formula is as follows: In the formula, To coordinate the scheduling of risk indices, As a risk factor for abnormal scenarios, For load phase sensitivity factors, This is the loss amplification factor. , , These are the weighting coefficients for the corresponding risk dimensions.

[0172] The weighting coefficients are set using the analytic hierarchy process (AHP) combined with scheduling strategy preferences. When prioritizing the risk of abnormal scenarios, the weighting coefficients can be set accordingly. , , .

[0173] The collaborative scheduling risk index is divided into multiple risk levels based on a preset numerical range, and the risk characteristics and control requirements corresponding to each risk level are determined to obtain quantitative data on collaborative scheduling risks.

[0174] The risk levels are divided into four levels: low risk, medium risk, high risk, and extremely high risk, with corresponding collaborative scheduling risk index ranges, such as low risk 0 to 0.3, medium risk 0.3 to 0.6, high risk 0.6 to 0.8, and extremely high risk 0.8 to 1.0.

[0175] The risk characteristics and control requirements corresponding to each risk level are as follows: Low risk indicates that the system is operating well and no active intervention is required; the current scheduling strategy can be maintained. Medium risk indicates that the system has a potential tendency to be abnormal and the monitoring frequency needs to be increased and an emergency scheduling plan needs to be prepared. High risk indicates that the system has shown obvious abnormalities and the scheduling parameters need to be adjusted immediately to mitigate the risk. Very high risk indicates that the system faces the risk of serious performance degradation or business interruption and emergency scheduling strategies need to be implemented, including measures such as task migration, traffic restriction, and hardware isolation.

[0176] The scheduling and coordination risk quantification data includes abnormal scenario types, coordination scheduling risk index, risk level, risk characteristics and control requirements, and is stored in association by resource domain and time slice.

[0177] An initial dynamic storage scheduling strategy is generated based on preset scheduling control requirements.

[0178] The system throughput target, IO response latency constraint, and system energy consumption threshold are taken as scheduling control requirements, broken down into target values ​​of specific scheduling parameters, and weight coefficients of each target are set according to business priority.

[0179] The system throughput target represents the total number of I / O operations that the system needs to complete per unit of time; the I / O response latency constraint represents the maximum allowable value of average I / O response latency; and the system energy consumption threshold represents the maximum allowable value of the total power consumption of the storage scheduling system.

[0180] The adjustment control requirements are broken down into target values ​​for specific scheduling parameters, including CPU core scheduling frequency, upper limit of IO queue depth, upper limit of data migration bandwidth, cache allocation ratio, and storage media placement strategy parameters.

[0181] For example, the system throughput target of 80,000 IOPS can be broken down into throughput targets for each resource domain. The four local resource domains are responsible for 20,000 IOPS, 18,000 IOPS, 18,000 IOPS and 14,000 IOPS respectively, and the cross-node shared domain is responsible for 10,000 IOPS. The IO response latency constraint of 500 microseconds can be broken down into response latency targets for each storage layer. The target for the NVMe layer is 100 microseconds, the target for the SATA layer is 5 milliseconds, and the target for the memory layer is 100 nanoseconds.

[0182] The weight coefficients of each target are set according to the business priority. The business priority is divided into three levels: critical business, important business and ordinary business. The throughput and latency targets of critical business have higher weights. In this embodiment, the throughput weight of critical business is set to 0.5, the latency weight to 0.4 and the energy consumption weight to 0.1, and the throughput weight to 0.3, the latency weight to 0.3 and the energy consumption weight to 0.4.

[0183] Based on historical scheduling data, computing power allocation parameters, IO scheduling parameters, and data migration parameters are selected as key scheduling parameters, and a target correlation model between key scheduling parameters and scheduling target values ​​is established.

[0184] Historical scheduling data represents a paired dataset of scheduling parameter settings and corresponding measured performance indicators collected during past operating cycles.

[0185] The computing power allocation parameters include CPU core binding strategy, core frequency adjustment level and interrupt affinity settings; the IO scheduling parameters include IO scheduling algorithm selection, queue depth configuration, request merging threshold and timeout retries; the data migration parameters include cold and hot data migration threshold, migration bandwidth limit, migration concurrency and migration time window.

[0186] The method for selecting key scheduling parameters is to calculate the correlation coefficient between each parameter and the scheduling target value based on historical data, and select parameters whose absolute value of the correlation coefficient is greater than a preset threshold (such as 0.5) as key scheduling parameters.

[0187] The target association model describes the quantitative mapping relationship between key scheduling parameters and scheduling target values. It is constructed using a gradient boosting tree model. The model input is a combination of key scheduling parameters, and the output is the predicted values ​​of system throughput, IO response latency, and system energy consumption.

[0188] The model training process is as follows: the historical dataset is divided into a training set and a validation set in chronological order, with the training set accounting for 80% and the validation set accounting for 20%; feature engineering is performed on the training set data, including parameter normalization, cross-feature construction, and temporal feature extraction; the hyperparameters of the gradient boosting tree are set to 200 trees, a maximum depth of 6, and a learning rate of 0.05; the model is fitted using the training set, and the prediction accuracy is evaluated on the validation set after each iteration; training is stopped early when the prediction error on the validation set no longer decreases for 10 consecutive iterations.

[0189] After the model is trained, the prediction accuracy is evaluated using a validation set. The throughput prediction error is required to be no more than 5%, and the latency prediction error is required to be no more than 10%.

[0190] Based on the business load phase, parameter threshold ranges for each phase are set in conjunction with the target values ​​of scheduling parameters, and a preliminary scheduling strategy is generated according to the linkage rules between different parameters.

[0191] The parameter threshold range represents the allowed value range of each key scheduling parameter under the current load stage. The lower limit and upper limit correspond to the boundaries of performance guarantee and resource protection, respectively.

[0192] For example, during high load periods, the threshold range for CPU core frequency adjustment is 80% to 100% of the highest frequency, the threshold range for the upper limit of IO queue depth is 16 to 64, and the threshold range for the upper limit of data migration bandwidth is 10% to 30% of the total bandwidth.

[0193] The linkage rules between different parameters indicate the related parameters that need to be adjusted synchronously and the direction of adjustment when adjusting parameters. When increasing the upper limit of IO queue depth, the number of CPU interrupt handling cores needs to be increased synchronously to avoid interrupt backlog. When decreasing the upper limit of data migration bandwidth, the cold and hot data migration cycle needs to be extended synchronously to ensure the data placement effect.

[0194] The initial scheduling strategy is a set of parameter settings for each load stage. Each parameter setting is the median value of the corresponding threshold range or the optimal value obtained by optimization based on the target association model. The strategy is configured separately for each load stage, and the system automatically switches the corresponding strategy according to the real-time load stage.

[0195] When switching scheduling strategies, a smooth transition mechanism is adopted. A transition time window is set between the old and new strategies. Within the transition window, the parameter values ​​gradually change from the old values ​​to the new values ​​by linear interpolation, so as to avoid system oscillation caused by sudden parameter changes.

[0196] The initial scheduling strategy is validated from three dimensions: system throughput, IO response latency, and system energy consumption. Multi-objective optimization is performed according to the target weight coefficients to obtain the initial dynamic storage scheduling strategy.

[0197] The verification method involves inputting the parameter settings of the initial scheduling strategy into the target association model, predicting the performance index values ​​in three dimensions, and comparing them with the scheduling control requirements.

[0198] Multi-objective optimization uses a weighted summation method to transform the three-dimensional objectives into a single comprehensive objective function. The comprehensive objective value is the weighted sum of the degree of achievement of each dimension, and the optimization objective is to maximize the comprehensive objective value. The calculation formula is as follows: In the formula, For the comprehensive target value, To predict throughput, For throughput target, To predict response latency, In response to latency targets, To predict energy consumption, Energy consumption threshold , , These are the target weight coefficients for the corresponding dimensions.

[0199] The optimization process searches for the parameter combination that maximizes the overall objective value within the threshold range of each parameter. The particle swarm optimization algorithm is used, and its execution flow is as follows: 50 particles are randomly initialized with positions and velocities within the parameter threshold range, with each particle's position representing a set of scheduling parameters; the position of each particle is input into the objective association model to calculate the overall objective value as fitness; the historical best position and global best position of each particle are recorded; and the particle positions are adjusted according to the particle velocity update formula, which is: ,in For inertial weights, , As a learning factor, , The number is a random number between 0 and 1; the number of iterations is set to 100. When the change in the global optimum is less than 0.001 for 20 consecutive iterations, convergence is determined and the process is terminated early.

[0200] The parameter combination obtained after optimization is the initial dynamic storage scheduling strategy, which achieves optimal overall performance while satisfying constraints in all dimensions.

[0201] Based on the scheduling coordination risk quantification data, the set of parameters to be adjusted is marked in the initial dynamic storage scheduling strategy, and the optimized scheduling strategy is obtained by performing parameter iterative correction.

[0202] We analyzed the impact parameters corresponding to the scheduling coordination risk quantification data of different storage scheduling anomaly scenarios, determined the risk mitigation logic of parameter adjustment, and constructed a risk-parameter association rule base.

[0203] The influencing parameters are scheduling parameters whose changes in value can have a significant impact on the collaborative scheduling risk index, and are determined through sensitivity analysis.

[0204] Adjusting the direction and magnitude of a parameter's value can reduce the causal relationship of risks in specific abnormal scenarios. For scenarios with computing power overload, reducing the CPU core allocation ratio for non-critical business or enabling core frequency downgrading can alleviate computing power pressure. For scenarios with IO bottlenecks, increasing the IO queue depth or enabling IO request merging can improve IO processing capabilities. For scenarios with cache failure, increasing the cache allocation ratio for high-hot data or adjusting the cache replacement strategy can improve the cache hit rate.

[0205] The risk-parameter association rule base is indexed by the type of abnormal scenario. Each record includes the type of abnormal scenario, a list of influencing parameters, the recommended adjustment direction for each parameter, and an explanation of the mitigation logic. The rule base is built based on historical scheduling experience and domain knowledge.

[0206] The rule base is constructed as follows: extract parameter adjustment records and risk change data before and after the occurrence of abnormal scenarios from historical scheduling logs; use association rule mining algorithms such as the Apriori algorithm to calculate the confidence and support between parameter adjustment and risk reduction; select rules with confidence greater than 0.7 and support greater than 0.1 to add to the rule base; label each rule with a verified mitigation effect level, divided into three levels: significant, moderate and slight.

[0207] For each storage scheduling anomaly scenario, calculate the sensitivity coefficient of each influencing parameter.

[0208] The sensitivity coefficient represents the change in the collaborative scheduling risk index caused by a unit parameter adjustment. The larger the absolute value of the sensitivity coefficient, the more significant the effect of the parameter on risk control.

[0209] The sensitivity coefficient is calculated by applying a small perturbation to a single influencing parameter based on the initial scheduling strategy and observing the change in the cooperative scheduling risk index. The calculation formula is as follows: In the formula, For the first The first control parameter affects the first Sensitivity coefficient for anomaly risk For the first The change in the risk index. For the first The adjustment amount of each control parameter.

[0210] For example, when the IO queue depth increases from 32 to 40 (an adjustment of 8), the risk index of the computing power overload scenario decreases from 0.6 to 0.54 (a change of -0.06). Therefore, the sensitivity coefficient of this parameter to the risk of computing power overload is -0.0075. The negative sign indicates that increasing this parameter can reduce the risk.

[0211] Based on the risk-parameter association rule base, the parameter categories, adjustment ranges, and adjustment priorities that need to be adjusted are marked in the initial dynamic storage scheduling policy to obtain the set of parameters to be adjusted.

[0212] The parameter categories are divided into computing power allocation, IO scheduling, and data migration categories according to the type of scheduling parameter.

[0213] The adjustment range represents the minimum and maximum values ​​that the parameter can be adjusted to. It is determined by the parameter threshold range and the current risk level. A larger adjustment range is allowed at higher risk levels.

[0214] The priority adjustment is determined based on the absolute value of the sensitivity coefficient and the urgency of risk mitigation. The larger the absolute value of the sensitivity coefficient and the higher the risk level, the higher the priority. The priority is divided into three levels: high, medium and low, with high priority parameters being adjusted first.

[0215] The set of parameters to be adjusted is the union of the influencing parameters corresponding to all currently identified abnormal scenarios. For parameters that affect multiple abnormal scenarios simultaneously, the intersection or weighted comprehensive direction of the adjustment directions under each scenario is taken as the final adjustment direction. If the adjustment directions conflict, the scenario with the higher risk level is given priority.

[0216] Based on the mitigation logic, determine the direction of parameter adjustment, calculate the adjustment range by combining the risk level and sensitivity coefficient, design the sequence of parameter adjustments for multi-parameter linkage adjustment scenarios, and generate parameter adjustment schemes.

[0217] The direction of parameter adjustment is determined by the mitigation logic, that is, it is adjusted in the direction of reducing the risk index of coordinated scheduling.

[0218] The adjustment magnitude is calculated by comprehensively considering the risk level and sensitivity coefficient. Risk mitigation adjustment is only performed when the current collaborative scheduling risk index is higher than the target risk index. If the current risk is lower than the target risk index, the adjustment magnitude is 0, meaning no adjustment is performed. The formula for calculating the adjustment magnitude is: In the formula, For the first The adjustment range of each parameter, The current risk index for coordinated scheduling. The target risk index is set at 0.3. This parameter represents the sensitivity coefficient to the risk of the current dominant anomaly scenario. This is a risk level adjustment coefficient, with 0.2 for low risk, 0.5 for medium risk, 0.8 for high risk, and 1.0 for very high risk.

[0219] In scenarios involving the coordinated adjustment of multiple parameters, the order of parameter adjustments follows the principle of adjusting high-priority parameters first and then low-priority parameters. At the same time, the dependencies between parameters are considered, with dependent parameters being adjusted first.

[0220] If interrupt affinity needs to be adjusted synchronously after adjusting CPU core allocation, then CPU core allocation should be adjusted first, followed by interrupt affinity.

[0221] The parameter adjustment plan includes the adjustment direction, adjustment range, adjustment sequence, and expected risk mitigation effect for each parameter to be adjusted.

[0222] Substitute the parameter adjustment scheme into the initial dynamic storage scheduling strategy, and determine whether it comprehensively meets the scheduling control requirements. If it does, generate an optimized scheduling strategy; otherwise, correct the adjustment magnitude and direction and re-verify.

[0223] The criteria for comprehensively meeting the scheduling and control requirements are as follows: the system throughput predicted by the adjusted strategy through the target correlation model is not lower than the preset percentage of the target value, such as 95%; the predicted IO response latency is not higher than the preset percentage of the target value, such as 110%; and the predicted system energy consumption is not higher than the energy consumption threshold.

[0224] If all three dimensions are satisfied, the parameter adjustment scheme is applied to the initial dynamic storage scheduling strategy to generate an optimized scheduling strategy. If any dimension is not satisfied, the adjustment magnitude and direction are corrected. Specifically, the adjustment magnitude of the parameter that causes performance to exceed the standard is reduced or its adjustment direction is reversed. Then, the parameter is re-substituted and verified. The maximum number of iterations is set to 10. If the maximum number of iterations is reached and the target value is still not met, the parameter combination with the highest comprehensive target value is taken as the optimized scheduling strategy.

[0225] The scheduling balance index is a real-time data collection and scheduling status data of each resource domain after execution.

[0226] After the scheduling is executed, the coupled status data of each resource domain is collected. Based on the status change characteristics of each parameter, a differentiated collection frequency is set, and the data is normalized to obtain the scheduling processing data.

[0227] Different sampling intervals are set according to the time-varying characteristics of the parameters. Parameters that change rapidly are sampled at a higher frequency, while parameters that change slowly are sampled at a lower frequency.

[0228] For example, if CPU computing power utilization changes rapidly, the sampling frequency is set to 1 millisecond; if IO throughput characteristics change moderately, the sampling frequency is set to 10 milliseconds; if data access popularity levels change slowly, the sampling frequency is set to 5 seconds; and if hardware wear and tear status changes the slowest, the sampling frequency is set to 1 hour.

[0229] The normalization process uses the min-max normalization method to map each parameter value to the range of 0 to 1.

[0230] The scheduling data consists of normalized coupled state data of each resource domain, which are aligned according to a unified time slice to form the input for evaluating the scheduling effect.

[0231] The mean, standard deviation, and coefficient of variation of the same parameter are calculated for each resource domain of the entire system. For a single resource domain, the fluctuation amplitude and frequency of the parameter within a continuous acquisition period are calculated to obtain the single-index equilibrium characteristics.

[0232] The mean of the same parameter in each resource domain of the entire system is the arithmetic mean of the values ​​of that parameter in all resource domains. The standard deviation reflects the degree of dispersion of the parameter values ​​in each resource domain relative to the mean. The coefficient of variation is the ratio of the standard deviation to the mean, eliminating the influence of dimensions for cross-parameter comparisons.

[0233] For a single resource domain, the parameter fluctuation amplitude within a continuous acquisition period is the difference between the maximum and minimum values ​​of the parameter within that period. The fluctuation frequency is the number of times the parameter value crosses the mean divided by the period length, reflecting the severity of the parameter oscillation.

[0234] The single-index balance characteristics include the inter-domain mean, inter-domain standard deviation, inter-domain coefficient of variation, intra-domain fluctuation amplitude, and intra-domain fluctuation frequency of each parameter, which are the basic characteristic quantities for calculating the scheduling balance degree.

[0235] Based on the single-index balance characteristics, and combining the two dimensions of inter-domain consistency and temporal stability, corresponding weights are assigned to calculate the single-parameter scheduling balance.

[0236] Inter-domain consistency reflects the uniformity of parameter distribution across resource domains, represented by 1 minus the truncated coefficient of variation. The coefficient of variation is truncated at an upper limit, taking the smaller value between the coefficient of variation and 1, to avoid a negative balance due to a coefficient of variation greater than 1. Temporal stability reflects the smoothness of parameter changes over time within a single resource domain, represented by 1 minus the normalized fluctuation amplitude. Smaller fluctuation amplitude indicates higher temporal stability. Single-parameter scheduling balance is the weighted sum of the scores from the two dimensions, calculated using the following formula: In the formula, For the first The coefficient of variation of each parameter The truncated coefficient of variation. Assigning a score to the inter-domain consistency score. For the first The actual fluctuation range of each parameter This parameter represents the maximum allowable fluctuation range. For time-domain stability score, , These are the weight coefficients for the corresponding dimensions. For the first Single-parameter scheduling balance of each parameter.

[0237] The weighting coefficient is set according to the scheduling objective, with a preference for load balancing between resource domains. , When prioritizing system stability, set , .

[0238] The single-parameter scheduling balance score ranges from 0 to 1. The closer the value is to 1, the better the scheduling balance effect of that parameter.

[0239] By combining the weights of the impact of different parameters on storage scheduling performance, the corresponding single-parameter scheduling balance is weighted and summed to obtain the scheduling balance index.

[0240] Different parameters have varying degrees of impact on storage scheduling performance. CPU computing power utilization and IO response latency have a significant impact on system performance and are given higher weights; link bandwidth utilization and cache hit rate have a moderate impact and are given medium weights; data popularity distribution has a relatively small impact and is given lower weights.

[0241] The influence weights are determined by the entropy weight method. The weights are calculated based on the information entropy of each parameter in historical data. The smaller the information entropy, the higher the distinguishability of the parameter, and the greater the weight. The sum of all weights is 1.

[0242] The scheduling balance index is the weighted sum of the scheduling balance of all single parameters, and the calculation formula is: In the formula, As a scheduling balance index, The total number of parameters participating in the evaluation. For the first Single-parameter scheduling balance with individual parameters. For the first The weights of each parameter on the performance of storage scheduling, and satisfying the following conditions: .

[0243] The scheduling balance index ranges from 0 to 1, with a value closer to 1 indicating a better overall scheduling balance.

[0244] Each control node in the optimized scheduling strategy is comprehensively evaluated. Control nodes that fail to meet the evaluation criteria are defined as control failure nodes, and a correction and adjustment strategy is generated for the control failure nodes.

[0245] The optimized scheduling strategy is categorized and sorted by load stage, control resource domain, and parameter type. All control nodes are identified, and the parameter settings, control range, and control objects of each control node are determined to obtain node characteristic data.

[0246] A control node represents the smallest independently controllable parameter unit in an optimized scheduling strategy. Each node corresponds to a combination of a specific load phase, a specific resource domain, and a specific parameter type.

[0247] During high-load phases, the IO queue depth parameter of the first resource domain serves as a control node.

[0248] The parameter setting value is the target value of the node in the optimized scheduling strategy. The control range is the minimum and maximum value boundary allowed by the node. The control object is the hardware resource or software module that the parameter affects, such as the CPU core, storage controller, cache manager, etc.

[0249] The node characteristic data is indexed by the control node number and includes load stage label, resource domain number, parameter type, parameter setting value, lower limit of control range, upper limit of control range, and control object identifier, which constitute the basic information for node evaluation.

[0250] By using historical control data, the weight of the deviation impact of each control node on the scheduling balance index is calculated.

[0251] The deviation impact weight represents the degree of influence of the parameter of the control node on the overall scheduling balance index when it deviates from the set value. The greater the degree of influence, the higher the weight.

[0252] The calculation method is based on the correspondence between the parameter deviation of each node and the change of the scheduling balance index in the historical control data. Regression analysis is used to obtain the deviation influence coefficient of each node, and then normalized into weights.

[0253] If a 10% deviation in the parameters of one node causes a decrease of 0.05 in the scheduling balance index, and a 10% deviation in the parameters of another node causes a decrease of 0.02 in the scheduling balance index, then the former's deviation has a higher weight than the latter's.

[0254] The bias impact weight ranges from 0 to 1, and the sum of the weights of all nodes is 1. The weights are recalculated and updated quarterly based on the latest historical data.

[0255] For each control node, based on its node characteristic data and deviation impact weight, a comprehensive evaluation is conducted according to the preset scheduling balance index threshold and the preset node control deviation threshold to obtain the node evaluation result.

[0256] The preset scheduling balance index threshold represents the minimum standard that the overall scheduling balance of the system needs to achieve, and is set to 0.6.

[0257] The preset node control deviation threshold represents the maximum allowable relative deviation between the actual value and the set value of a single control node parameter, which is set to 15%.

[0258] The comprehensive evaluation process is as follows: Check whether the actual value of the node's parameter is within the control range. If it exceeds the range, the evaluation is directly deemed unsuccessful. Calculate the relative deviation between the actual value and the set value of the parameter. If it exceeds the node control deviation threshold, the evaluation is deemed unsuccessful. Then evaluate the impact of the node's deviation on the overall scheduling balance index. If the deviation impact weight multiplied by the deviation degree results in the overall balance being lower than the threshold, the evaluation is deemed unsuccessful. If all conditions are met, the evaluation is deemed successful.

[0259] The node evaluation results include the node number, the conclusion of whether the evaluation passed or failed, the specific reasons for failure, and the deviation quantification value.

[0260] Node failure degree is calculated based on node evaluation results, node failure level is divided according to preset value range, and nodes that reach the preset failure level are selected as control failure nodes.

[0261] Node failure rate comprehensively reflects the node assessment score, the weight of deviation impact, and the risk diffusion effect. The calculation formula is as follows: In the formula, For the first The failure rate of each control node Analyze the nodes using a percentage-based score. The reverse proportion of the node evaluation score. This represents the weight of the node's impact on the overall system scheduling performance. This represents the risk diffusion coefficient for node failure.

[0262] The node evaluation score is calculated based on the degree to which each condition in the evaluation is met. 100 points are awarded if all conditions are met. Points are deducted for each violation. 40 points are deducted for parameters exceeding the control range, 30 points are deducted for deviations exceeding the threshold, and 30 points are deducted for affecting the overall balance.

[0263] The risk diffusion coefficient indicates the degree to which the failure of a node may trigger a cascading failure of other nodes. It is positively correlated with the out-degree of the node in the scheduling dependency graph. The diffusion coefficient is set to 1.5 for core scheduling nodes, 1.0 for ordinary nodes, and 0.5 for edge nodes.

[0264] Node failure levels are classified according to the range of failure severity values: 0 to 0.2 indicates minor failure, 0.2 to 0.5 indicates moderate failure, 0.5 to 0.8 indicates severe failure, and 0.8 to 1.0 indicates critical failure.

[0265] The default failure level is set to moderate failure or above, meaning that nodes with a failure level greater than or equal to 0.2 are selected as control failure nodes.

[0266] For each control failure node, analyze the root cause of the control failure, and generate corresponding corrective adjustment strategies based on the problem type of the root cause and the failure level of the node.

[0267] The root causes of control failure can be categorized into four types: unreasonable parameter settings, hardware performance deviating from expectations, sudden changes in external load, and failure of related nodes.

[0268] Inappropriate parameter settings indicate that the parameter settings for this node in the optimized scheduling strategy do not match the actual system state, causing the parameters to fail to achieve the expected results; hardware performance deviates from expectations, indicating that the actual performance deviates significantly from the baseline performance due to hardware wear and tear or failure, rendering the parameter settings based on the baseline performance ineffective; sudden external load changes indicate that the business load suddenly exceeds the load phase range designed by the strategy, causing the original parameter settings to be unable to cope; associated node failure indicates that the control of this node depends on the normal output of other nodes, and the failure of associated nodes causes this node to be unable to work normally.

[0269] For different root causes and failure levels, corresponding corrective adjustment strategies are generated: For moderate failures caused by unreasonable parameter settings, the parameter optimization process is re-executed to update the settings of the node; For severe failures caused by hardware performance deviating from expectations, the load allocation ratio of the resource domain where the node is located is reduced and a hardware health check is triggered; For severe failures caused by sudden changes in external load, an emergency load migration strategy is executed to transfer some services to other resource domains; For failures caused by the failure of related nodes, the related nodes are repaired first and then the current node is re-evaluated.

[0270] The corrective adjustment strategy includes the failure node number, root cause analysis, specific adjustment measures, expected recovery time, and verification method. The strategy is issued and executed immediately after generation, and the scheduling balance index is re-collected after execution to verify the corrective effect.

[0271] In summary, this embodiment provides a dynamic storage scheduling method based on CPU computing power awareness. By comprehensively collecting all dimensions of system operating parameters and performing regular resource allocation, it achieves collaborative awareness and unified scheduling of computing and storage resources. It can match the corresponding data access load according to the computing supply capacity, improve the adaptability and overall utilization efficiency of the two types of resources, and alleviate the performance loss caused by uneven resource distribution.

[0272] By analyzing the correlation between various operating states, different abnormal situations can be identified. By combining the business load stage and hardware wear status, scheduling risks can be quantified, enabling early prediction and hierarchical control of risks. This transforms passive anomaly handling into proactive risk avoidance, reduces the impact of scheduling conflicts and performance degradation on business operations, and ensures the continuity of business operations.

[0273] An initial strategy is generated based on scheduling and control requirements, and parameters are iteratively corrected by combining risk quantification results. This enables the scheduling strategy to adapt to the current system state and risk level, achieving optimal scheduling performance under multiple constraints of performance, latency, and energy consumption, and ensuring long-term stable service quality of the system.

[0274] Based on the same general inventive concept, this invention also protects a CPU-power-aware dynamic memory scheduling system. The following describes a CPU-power-aware dynamic memory scheduling system provided by this invention. The CPU-power-aware dynamic memory scheduling system described below can be referred to in correspondence with the CPU-power-aware dynamic memory scheduling method and system described above.

[0275] A CPU-power-aware dynamic storage scheduling system includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The processor includes a multi-domain coupling monitoring module, a scheduling risk quantification and assessment module, a dynamic scheduling strategy generation module, and a control node feedback correction module.

[0276] The multi-domain coupling monitoring module is used to collect all-dimensional operating parameters of the storage scheduling system, divide the storage resource pool into resource domains, and collect data on CPU computing power margin, storage IO throughput characteristics, data access heat level, cache hit efficiency and interconnect link bandwidth utilization of each resource domain in real time as multi-dimensional coupling status data.

[0277] The scheduling risk quantification assessment module is used to identify different storage scheduling anomaly scenarios based on multi-dimensional coupled state data. It combines the current stage of business load and hardware wear status to construct scheduling collaborative risk quantification data under different storage scheduling anomaly scenarios.

[0278] The dynamic scheduling strategy generation module is used to generate an initial dynamic storage scheduling strategy based on preset scheduling control requirements. It marks the set of parameters to be adjusted in the initial dynamic storage scheduling strategy according to the scheduling coordination risk quantification data, and performs iterative parameter correction to obtain an optimized scheduling strategy.

[0279] The control node feedback correction module is used to collect the scheduling balance index of the coupling status data of each resource domain after scheduling execution in real time, to comprehensively evaluate each control node in the optimized scheduling strategy, to define the control node that fails to meet the evaluation as the control failure node, and to generate a correction adjustment strategy pointing to the control failure node.

[0280] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0281] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic memory scheduling method based on CPU computing power awareness, characterized in that, include: Collect all dimensions of the storage scheduling system's operating parameters, divide the storage resource pool into resource domains, and collect multi-dimensional coupling status data of each resource domain in real time; Based on the multi-dimensional coupled state data, different storage scheduling anomaly scenarios are identified. Combined with the current stage of business load and hardware wear status, scheduling coordination risk quantification data under different storage scheduling anomaly scenarios is constructed. An initial dynamic storage scheduling strategy is generated based on preset scheduling control requirements. The set of parameters to be adjusted is marked in the initial dynamic storage scheduling strategy according to the scheduling coordination risk quantification data. The optimized scheduling strategy is obtained by performing parameter iterative correction. The scheduling balance index of each resource domain coupling status data is collected in real time after scheduling execution. Each control node in the optimized scheduling strategy is comprehensively evaluated. Control nodes that fail to meet the evaluation are defined as control failure nodes, and a correction and adjustment strategy is generated for the control failure nodes.

2. The dynamic storage scheduling method based on CPU computing power awareness according to claim 1, characterized in that, The process of partitioning a storage resource pool into resource domains includes: The server NUMA node topology, CPU core group distribution location, storage medium array layout parameters, and business data deployment granularity are collected as the full-dimensional operating parameters. Based on the computing power-storage affinity requirements of business tasks and the target scheduling response latency, NUMA nodes are used as the basic unit to meet the condition of uniform computing power supply. The storage resource pool is regularly divided according to the distribution location of the storage medium array to determine the number of resource domains. The storage access boundary of each resource domain is defined based on the number of interconnection hops between the data block and the CPU core group.

3. The dynamic storage scheduling method based on CPU computing power awareness according to claim 1, characterized in that, The process of collecting the multi-dimensional coupled state data includes: Collect user-mode utilization and kernel-mode utilization of CPU core groups, calculate total computing power utilization and inter-domain computing power difference, and form CPU computing power margin data; Collect random read IOPS, sequential write IOPS and average IO response latency in each resource domain, and record IO queue depth and data block size. Normalize and correct according to standard data block size to obtain storage IO throughput characteristics. The access frequency and recent access time interval of data blocks in each resource domain are collected according to a preset frequency. The data heat value is obtained based on the heat calculation model. The heat distribution concentration and hot spot shift time period are analyzed to form a data access heat level. The cache hit count within each resource domain and the total number of accesses within the statistical period are statistically analyzed, and the cache hit efficiency is obtained based on the cache hit calculation formula. Measure the real-time data transmission rate of different storage tiers on the interconnect links of each resource domain, calculate the bandwidth utilization of the corresponding tier based on the rated bandwidth of the link, record the bandwidth differences of different tiers, and obtain interconnect link bandwidth utilization data. Time-series alignment is performed on CPU computing power margin, storage I / O throughput characteristics, data access popularity level, cache hit efficiency, and interconnect link bandwidth utilization data within the same resource domain. Correlation characteristics between different parameters are marked to obtain the multi-dimensional coupling state data.

4. The dynamic storage scheduling method based on CPU computing power awareness according to claim 1, characterized in that, The process of identifying different storage scheduling anomaly scenarios includes: The coupling state data of each resource domain is standardized and aligned in spatiotemporal dimension according to the resource domain number and time slice to obtain the coupling state processing data. From the coupled state processing data, extract single-parameter features and multi-parameter coupling features related to storage scheduling anomalies as scheduling anomaly features; Based on the service performance requirements of dynamic storage scheduling, three threshold levels are set for the aforementioned scheduling anomaly characteristics; An abnormal scenario matching rule base is established based on the three-level threshold and the scheduling anomaly characteristics. The coupling state processing data of each resource domain is traversed, and the corresponding storage scheduling anomaly scenario is determined by combining the matching rule base.

5. The dynamic storage scheduling method based on CPU computing power awareness according to claim 1, characterized in that, The process of constructing the scheduling and coordination risk quantification data includes: The absolute deviation of performance indicators relative to the target value for different storage scheduling anomaly scenarios is converted into relative deviation rate. Based on the sensitivity of different load stages to scheduling anomaly scenarios, the basic weight of the load stage is set, and the load stage sensitivity factor is calculated. The CPU computing performance degradation rate, disk read and write performance degradation rate, monitoring probe accuracy deviation rate, cache efficiency degradation rate and hardware cumulative runtime are normalized and weighted to obtain the hardware loss coefficient. For the aforementioned storage scheduling anomaly scenarios, risk factors for the anomaly scenarios are classified from three dimensions: the severity of the anomaly type, the scope of its impact, and the duration of its duration. Calculate the loss amplification factor based on the aforementioned hardware loss coefficient; The collaborative scheduling risk index is calculated by coupling the abnormal scenario risk factor, load stage sensitivity factor and loss amplification factor through a weighted summation method. The collaborative scheduling risk index is divided into multiple risk levels according to a preset numerical range, and the risk characteristics and control requirements corresponding to each risk level are determined to obtain the quantitative data of the collaborative scheduling risk.

6. The dynamic storage scheduling method based on CPU computing power awareness according to claim 1, characterized in that, The process of generating the initial dynamic storage scheduling policy includes: The system throughput target, IO response latency constraint, and system energy consumption threshold are taken as the scheduling control requirements, which are broken down into target values ​​of specific scheduling parameters, and weight coefficients of each target are set according to business priority. Based on historical scheduling data, computing power allocation parameters, IO scheduling parameters, and data migration parameters are selected as key scheduling parameters, and a target correlation model between key scheduling parameters and scheduling target values ​​is established. Based on the business load stage, the parameter threshold range for each stage is set in conjunction with the target value of the scheduling parameters, and a preliminary scheduling strategy is generated according to the linkage rules between different parameters. The initial scheduling strategy is validated from three dimensions: system throughput, IO response latency, and system energy consumption. Multi-objective optimization is performed according to the target weight coefficients to obtain the initial dynamic storage scheduling strategy.

7. The dynamic storage scheduling method based on CPU computing power awareness according to claim 6, characterized in that, The process of obtaining the optimized scheduling strategy includes: We analyzed the impact parameters corresponding to the scheduling coordination risk quantification data of different storage scheduling anomaly scenarios, determined the risk mitigation logic of parameter adjustment, and constructed a risk-parameter association rule base. For each storage scheduling anomaly scenario, calculate the sensitivity coefficient of each influencing parameter; Based on the risk-parameter association rule base, the parameter categories, adjustment ranges, and adjustment priorities that need to be adjusted are marked in the initial dynamic storage scheduling strategy to obtain the set of parameters to be adjusted; Based on the mitigation logic, the direction of parameter adjustment is determined, and the adjustment range is calculated by combining the risk level and sensitivity coefficient. For scenarios involving the coordinated adjustment of multiple parameters, the sequence of parameter adjustments is designed, and a parameter adjustment plan is generated. Substitute the parameter adjustment scheme into the initial dynamic storage scheduling strategy, and determine whether it comprehensively meets the scheduling control requirements. If yes, generate the optimized scheduling strategy; otherwise, correct the adjustment magnitude and direction and re-verify.

8. The dynamic storage scheduling method based on CPU computing power awareness according to claim 1, characterized in that, The process of obtaining the scheduling balance index includes: After the scheduling is executed, the coupling status data of each resource domain is collected. Based on the state change characteristics of each parameter, a differentiated collection frequency is set, and the data is normalized to obtain the scheduling processing data. Calculate the mean, standard deviation, and coefficient of variation of the same parameter in each resource domain of the entire system. For a single resource domain, calculate the parameter fluctuation amplitude and fluctuation frequency within a continuous acquisition period to obtain the single index equilibrium characteristics. Based on the single-index balance characteristics, and combined with the two dimensions of inter-domain consistency and temporal stability, corresponding weights are assigned to calculate the single-parameter scheduling balance degree. By combining the weights of the impact of different parameters on storage scheduling performance, the corresponding single-parameter scheduling balance is weighted and summed to obtain the scheduling balance index.

9. The dynamic storage scheduling method based on CPU computing power awareness according to claim 1, characterized in that, The process of generating the aforementioned correction and adjustment strategy includes: The optimized scheduling strategy is categorized and sorted into load phase, control resource domain, and parameter type. All control nodes are identified, and the parameter settings, control range, and control objects of each control node are determined to obtain node characteristic data. Calculate the weight of the deviation impact of each control node on the scheduling balance index using historical control data; For each control node, based on its node characteristic data and deviation impact weight, a comprehensive evaluation is conducted according to the preset scheduling balance index threshold and the preset node control deviation threshold to obtain the node evaluation result. The node failure degree is calculated based on the node evaluation results, the node failure level is divided according to the preset value range, and the nodes that reach the preset failure level are selected as the control failure nodes. For each control failure node, analyze the root cause of the control failure, and generate corresponding corrective adjustment strategies based on the problem type of the root cause and the failure level of the node.

10. A CPU-power-aware dynamic memory scheduling system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements a dynamic storage scheduling method based on CPU computing power awareness as described in any one of claims 1 to 9.

Images