AI-based computing platform management system

Abstract

This invention discloses an artificial intelligence-based computing platform management system, specifically relating to the field of dynamic allocation technology for distributed storage resources. It addresses the resource allocation oscillation problem caused by hardware performance differences and dynamic degradation in existing technologies. The system employs a real-time acquisition module to obtain the hardware performance, load status, and data paths of storage nodes; an entropy determination module to calculate the path topology entropy and trigger reconstruction instructions; a resonance analysis module to analyze the group correlation strength of hardware performance degradation and mark resonant node groups; a weight generation module to integrate the above parameters to generate dynamically allocated weights; an error compensation module to optimize weight coefficients based on historical error data; and a migration execution module to perform migrations based on the weights and limit the load on resonant nodes. Through the coordinated regulation of topology entropy monitoring and hardware resonance analysis, the system suppresses the spread of local oscillations, reduces unnecessary migration operations, and improves system stability and hardware lifespan.

Description

Technical Field

[0001] This invention relates to the field of distributed storage resource dynamic allocation technology, and more specifically, to an artificial intelligence-based computing platform management system. Background Technology

[0002] In distributed storage systems, storage nodes are typically composed of various heterogeneous hardware components, such as high-performance solid-state drives (SSDs), high-capacity hard disk drives (HDDs), and memory media. These hardware components differ significantly in performance metrics such as read / write speeds, latency, and lifespan. To improve resource utilization, existing technologies generally employ dynamic allocation strategies, adjusting data distribution based on the real-time load status of nodes (such as remaining storage space and I / O pressure). However, in environments where hardware performance varies significantly and dynamically, traditional resource allocation methods, relying on static rules or single-dimensional load metrics, struggle to adapt to complex performance fluctuation scenarios.

[0003] In existing technologies, due to differences in storage node hardware performance and dynamic degradation characteristics, resource allocation strategies are easily affected by the coupling interference of multiple dimensions. For example, when the hardware performance of a node (such as SSD write lifespan) undergoes a sudden change, traditional dynamic allocation methods cannot effectively coordinate the relationship between performance degradation and real-time load, leading to a continuous accumulation of prediction errors and frequent fluctuations in resource allocation results. This fluctuation manifests as unnecessary and repeated data migration between nodes, which not only reduces system efficiency but also exacerbates hardware wear and tear, affecting the overall stability of the storage service. Summary of the Invention

[0004] In order to overcome the above-mentioned deficiencies of the prior art, embodiments of the present invention provide an artificial intelligence-based computing platform management system to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] The AI-based computing platform management system includes the following modules:

[0007] Real-time acquisition module: Acquires hardware performance indicators, real-time load status and data access paths of each storage node in the distributed storage system in real time;

[0008] Entropy determination module: Calculates the topology entropy value of the data access path. When the topology entropy value exceeds the preset threshold, it is determined to be a chaotic migration path and triggers a path reconstruction command.

[0009] Resonance Analysis Module: Based on hardware performance indicators, analyzes the nonlinear resonance correlation strength of the hardware performance degradation rate between different storage nodes. When the nonlinear resonance correlation strength exceeds a critical threshold, the resonance node group is determined.

[0010] Weight generation module: Based on the nonlinear resonance correlation strength, resonance node group and path reconstruction instructions, it generates dynamic weights for nodes to suppress oscillations through a dynamic feature weighting model;

[0011] Error compensation module: Extracts prediction error data caused by sudden changes in hardware performance and path chaos in historical resource allocation, and generates error compensation coefficients by combining the dynamic weight allocation of nodes.

[0012] Migration Execution Module: Performs data migration operations across storage nodes based on the error compensation coefficient. The data migration operations restrict the allocation of high-frequency tasks in the resonant node group and prioritize the reconstruction of disordered paths.

[0013] In a preferred embodiment, the real-time acquisition of hardware performance indicators, real-time load status, and data access paths of each storage node in the distributed storage system includes:

[0014] The system collects the read / write speed, latency, and storage media lifespan of storage nodes as hardware performance indicators. The storage media lifespan is quantified by the remaining write cycles of solid-state drives and the cumulative runtime of hard disk drives. The system also collects the real-time load status of storage nodes, including remaining storage space and input / output pressure. Finally, the system collects the inter-node connectivity and data migration frequency of the data access path.

[0015] In a preferred embodiment, the topology entropy value of the data access path is calculated. When the topology entropy value exceeds a preset threshold, it is determined to be a chaotic migration path state and a path reconstruction instruction is triggered, including:

[0016] Based on the adjacency matrix of inter-node connections and the time series of data migration frequencies, the topological entropy value is calculated using the Shannon entropy formula. The Shannon entropy formula states that the topological entropy value is equal to the weighted sum of the information entropy of the probability distribution of inter-node connections and the coefficient of variation of the data migration frequency.

[0017] The topology entropy value is compared with a preset threshold, which is dynamically adjusted based on the upper bound of the historical topology entropy value statistical distribution.

[0018] When the topology entropy value exceeds a preset threshold, a migration path reconstruction instruction is generated. The migration path reconstruction instruction includes a list of path identifiers that need to be reconstructed first and the corresponding bandwidth occupancy constraints.

[0019] In a preferred embodiment, the coefficient of variation of data migration frequency is calculated by dividing the standard deviation of the time series of data migration frequency by the mean.

[0020] In a preferred embodiment, the nonlinear resonance correlation strength of the hardware performance degradation rate among different storage nodes is analyzed based on hardware performance indicators. When the nonlinear resonance correlation strength exceeds a critical threshold, a resonant node group is determined, including:

[0021] Extract the write life degradation rate of solid-state drives and the abnormal increase in seek time of mechanical hard drives from each storage node as time-series data of hardware performance degradation rate.

[0022] Construct a covariance matrix of hardware performance degradation rate among different nodes, and extract the maximum eigenvalue of the covariance matrix as the nonlinear resonance correlation strength using principal component analysis.

[0023] The nonlinear resonance correlation strength is compared with a critical threshold, which is dynamically set based on the statistical distribution characteristics of the eigenvalues ​​of the covariance matrix.

[0024] When the nonlinear resonance correlation strength exceeds the critical threshold, nodes whose eigenvector weights in the covariance matrix exceed a preset ratio are marked as resonance node groups.

[0025] In a preferred embodiment, based on the nonlinear resonance correlation strength, resonance node group, and path reconstruction instructions, a dynamic feature weighting model is used to generate dynamic node allocation weights to suppress oscillations, including:

[0026] The nonlinear resonance correlation strength is mapped to a suppression coefficient through a normalization function. The normalization function ensures that the suppression coefficient is positively correlated with the nonlinear resonance correlation strength.

[0027] The path reconstruction factor is generated based on the bandwidth occupancy constraints in the path reconstruction instruction. The path reconstruction factor is the product of the reciprocal of the topology entropy value of the path that needs to be reconstructed first and the remaining available bandwidth.

[0028] Dynamic weight allocation for nodes is generated based on a linear combination of inhibition coefficient, path reconstruction factor, and real-time load status.

[0029] In a preferred embodiment, the weighting coefficients of the linear combination are optimized using gradient descent based on historical resource allocation error data.

[0030] In a preferred embodiment, prediction error data caused by sudden changes in hardware performance and path chaos in historical resource allocation is extracted, and error compensation coefficients are generated by combining them with dynamically allocated node weights, including:

[0031] Extract the number of data migration errors caused by a sudden drop in the write lifespan of the solid-state drive from the resource allocation log, and the number of invalid migration operations caused by path chaos as historical prediction error data;

[0032] The mean of the sliding time window for the number of data migration errors and the number of invalid migration operations is calculated and used as the standardized historical prediction error data.

[0033] The standardized historical prediction error data and the dynamically assigned node weights are input into the error compensation function to generate error compensation coefficients.

[0034] In a preferred embodiment, the error compensation function is a nonlinear mapping relationship based on the hyperbolic tangent function, ensuring that the error compensation coefficient monotonically decreases as the product of historical prediction error data and node dynamic weight allocation decreases.

[0035] In a preferred embodiment, a data migration operation across storage nodes is performed based on an error compensation coefficient. This data migration operation restricts the allocation of high-frequency tasks to the resonant node group and prioritizes the reconstruction of disordered paths, including:

[0036] The task allocation priority of each storage node is adjusted according to the error compensation coefficient, and the task allocation priority is negatively correlated with the error compensation coefficient.

[0037] A high-frequency task allocation ratio limit is imposed on the resonant node group. The high-frequency task allocation ratio limit is dynamically set according to the ratio of the node's dynamic allocation weight to its real-time load status.

[0038] Based on the path identifier list in the migration path reconstruction instruction, data blocks stored in the paths that need to be reconstructed should be migrated first. The data block migration process follows bandwidth occupancy constraints and updates the topology entropy value in real time.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. By employing a synergistic mechanism of topology entropy monitoring and nonlinear resonance analysis, a dynamic resource allocation system linking hardware performance degradation and path status is constructed, significantly improving the stability and resource utilization of the distributed storage system. Through real-time quantification of the group correlation strength between path topology disorder and hardware performance degradation, the system can accurately identify oscillation sources and dynamically adjust weight allocation: Topology entropy monitoring identifies disordered paths based on node connection relationships and migration frequency, triggering reconstruction instructions to optimize data distribution and block local oscillations caused by path disorder; Nonlinear resonance analysis analyzes the synergistic effect of hardware performance degradation through the covariance matrix, limiting the high-frequency load of resonance node groups and suppressing the spread risk of hardware degradation hotspots; Both achieve joint regulation through a dynamic feature weighting model, forming a bidirectional negative feedback loop between path order and hardware status, fundamentally reducing unnecessary data migration, lowering hardware losses, and improving task allocation efficiency.

[0041] 2. Through a closed-loop design of error compensation mechanism and dynamic priority adjustment, adaptive optimization of resource allocation strategy is achieved; the error compensation coefficient is combined with historical prediction error and real-time weight to dynamically correct migration decisions, ensuring that the compensation logic strictly matches the system disturbance frequency; the high-frequency task allocation ratio limit is based on the dynamic constraint of the ratio of node weight and load status to prevent overloaded nodes from aggravating performance degradation due to task accumulation; the coordinated control of path priority migration and bandwidth occupancy constraints ensures critical path reconstruction while avoiding network congestion, taking into account both topology optimization and business continuity. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the structure of the AI-based computing platform management system of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0044] Example: Figure 1 A schematic diagram of the structure of the AI-based computing platform management system of the present invention is provided. The AI-based computing platform management system includes the following modules:

[0045] Real-time acquisition module: Acquires hardware performance indicators, real-time load status and data access paths of each storage node in the distributed storage system in real time;

[0046] Entropy determination module: Calculates the topology entropy value of the data access path. When the topology entropy value exceeds the preset threshold, it is determined to be a chaotic migration path and triggers a path reconstruction command.

[0047] Resonance Analysis Module: Based on hardware performance indicators, analyzes the nonlinear resonance correlation strength of the hardware performance degradation rate between different storage nodes. When the nonlinear resonance correlation strength exceeds a critical threshold, the resonance node group is determined.

[0048] Weight generation module: Based on the nonlinear resonance correlation strength, resonance node group and path reconstruction instructions, it generates dynamic weights for nodes to suppress oscillations through a dynamic feature weighting model;

[0049] Error compensation module: Extracts prediction error data caused by sudden changes in hardware performance and path chaos in historical resource allocation, and generates error compensation coefficients by combining the dynamic weight allocation of nodes.

[0050] Migration Execution Module: Performs data migration operations across storage nodes based on the error compensation coefficient. The data migration operations restrict the allocation of high-frequency tasks in the resonant node group and prioritize the reconstruction of disordered paths.

[0051] The AI-based computing platform management system achieves intelligent resource allocation decisions through multi-dimensional data perception and adaptive optimization of dynamic models. The system relies on real-time data collection of hardware performance, path status, and load indicators to construct a dynamic feature weighting model and error compensation mechanism. This simulates the multi-factor trade-offs and experience learning in human decision-making: the dynamic feature weighting model analyzes the correlation between hardware attenuation resonance intensity and path disorder to generate an optimal weight allocation strategy to suppress oscillations, simulating the comprehensive judgment in expert experience; the error compensation module iteratively corrects weight parameters based on historical error data, achieving strategy optimization similar to reinforcement learning, enabling the system to approach the global optimum during continuous operation; topological entropy monitoring and nonlinear resonance analysis serve as the core perception layer, providing precise input for upper-level intelligent decision-making. Finally, the transfer execution module controls resource scheduling in a closed loop, forming a complete AI control chain from data perception to autonomous decision-making.

[0052] The specific implementation method for real-time acquisition of hardware performance indicators, real-time load status, and data access paths of each storage node in a distributed storage system is as follows: The read and write speeds of storage nodes are collected using a standard test instruction set, which includes sequential and random read / write operations. Measurement results are recorded in megabytes per second. Latency is obtained by calculating the average time interval between storage node responses to input / output requests, with the time interval in milliseconds. The remaining erase / write cycles are obtained by parsing the output of the SSD's intelligent monitoring tool. The remaining erase / write cycles represent the number of erase / write operations the SSD can withstand remaining. The intelligent monitoring tool is a self-monitoring, analysis, and reporting technology tool. The cumulative runtime of the HDD is obtained by reading the cumulative running hours recorded in the HDD firmware. The cumulative running hours are accumulated from the first use of the hard drive.

[0053] The remaining storage space is obtained by calling the storage management interface of the distributed storage system. The storage management interface returns the currently unused storage capacity, in gigabytes. Input / output pressure is obtained by monitoring the input / output operation queue depth of the storage nodes. Queue depth represents the number of input / output requests waiting to be processed. Queue depth is collected in real time by the operating system performance monitoring tool and converted into a pressure index. The mapping rule between the pressure index and queue depth is as follows: when the queue depth is less than or equal to a preset threshold, the pressure index equals the queue depth divided by the preset threshold; when the queue depth is greater than the preset threshold, the pressure index is a fixed maximum value of 1.

[0054] The connection relationships between nodes are obtained by parsing the routing configuration table of the distributed storage system. The routing configuration table records the physical or logical connection topology between storage nodes. During parsing, invalid connection paths with the heartbeat detection protocol type are filtered out. The data migration frequency is obtained by counting the number of data block migration operations within a 5-minute sliding time window. The migration operation count is extracted from the storage system operation logs and aggregated using an open-source log processing framework. The step size of the sliding time window is 1 minute.

[0055] Hardware performance metrics, real-time load status, and data access path data are encapsulated into structured data packets using key-value pairs. These key-value pairs include read / write speed, latency, remaining erase / write cycles, cumulative runtime, remaining storage space, input / output pressure index, and data migration frequency. The structured data packets are transmitted to the resource allocation decision module via an Advanced Message Queuing Protocol (AMQP), which configures persistent storage and transmission acknowledgment mechanisms. Data is standardized using a normalization formula: the original value minus the minimum value, divided by the difference between the maximum and minimum values. The minimum and maximum values ​​are dynamically set based on the statistical distribution of historical data over the past 30 days; for example, the maximum read / write speed is 1000 megabytes per second, and the minimum is 0 megabytes per second.

[0056] Data validity is ensured by performing integrity checks on standardized data. Integrity checks include verifying field completeness and numerical reasonableness. The rules for determining numerical reasonableness are as follows: For solid-state drives (SSDs), the remaining write cycles must be greater than or equal to 0 and less than or equal to the manufacturer's stated maximum write cycles (e.g., a certain model of SSD has a maximum write cycle count of 3000); for hard disk drives (HDDs), the cumulative operating hours must be greater than or equal to 0 and less than or equal to the design life (e.g., a certain model of HDD has a design life of 100,000 hours). A retransmission request mechanism handles data anomalies. When data is missing or exceeds limits, the retransmission request mechanism sends a retransmission command containing an anomaly identifier and a timestamp to the data acquisition module. Upon receiving the retransmission command, the data acquisition module re-acquires data within a specified time window. After three failed retransmissions, nodes are marked as temporarily unavailable to exclude abnormal nodes. Nodes in a temporarily unavailable state do not participate in resource allocation until new valid data is received.

[0057] The specific implementation method for calculating the topology entropy of data access paths and triggering path reconstruction instructions is as follows: The storage node topology is constructed based on an adjacency matrix of inter-node connections. The adjacency matrix is ​​a two-dimensional matrix, where each element indicates whether a direct connection exists between two nodes. Elements with a direct connection have a value of 1, otherwise 0. The adjacency matrix is ​​constructed by parsing the routing configuration table of the distributed storage system. The routing configuration table records the physical or logical connection information between storage nodes. During parsing, invalid connection paths used only for heartbeat detection are filtered out; for example, paths with heartbeat detection as their protocol type are excluded.

[0058] The activity level of a path is quantified based on the time series of data migration frequency. The time series of data migration frequency consists of the number of path migration operations recorded in chronological order, with a time granularity of 5 minutes. The number of migration operations is extracted from the operation logs of the storage system using an open-source log processing framework. The log processing framework aggregates the number of migration operations within a 5-minute window according to the path identifier, generating key-value pairs with the path as the key and the number of migrations as the value.

[0059] The information entropy of the probability distribution of connections between nodes is calculated using the Shannon entropy formula. Information entropy quantifies the degree of disorder in the topological structure. The probability distribution of connections between nodes represents the probability of each node being connected to every other node in the adjacency matrix. The connection probability equals the number of direct connections between nodes divided by the total number of nodes minus 1. The information entropy is calculated by taking the logarithm of the connection probability for each node and then summing the results with weights equal to the connection probability itself. The coefficient of variation quantifies the time-series volatility of data migration frequency. The coefficient of variation of data migration frequency equals the standard deviation of the time series of data migration frequencies divided by the mean. The standard deviation and mean are calculated based on the number of migrations in the past 24 hours.

[0060] The information entropy and coefficient of variation are weighted and summed according to preset weights to obtain the topology entropy value. The preset weights are dynamically adjusted based on path stability requirements. For example, for storage paths with high stability requirements, the information entropy weight is set to 0.7 and the coefficient of variation weight is set to 0.3; for paths with low stability requirements, the information entropy weight is set to 0.5 and the coefficient of variation weight is set to 0.5. The weight adjustment rules are defined through a configuration file, which is set by the system administrator according to business needs.

[0061] The topology entropy value is compared with a preset threshold, which is dynamically adjusted based on the upper bound of the historical topology entropy value statistical distribution. The historical topology entropy value statistical distribution is the set of entropy values ​​of all paths within the same time period over the past 7 days. The upper bound is determined by calculating the mean of the historical entropy values ​​plus three times the standard deviation. For example, if the historical entropy value mean is 0.6 and the standard deviation is 0.15, then the upper bound is 0.6 plus 3 times 0.15, which equals 1.05. The dynamic adjustment process is performed every 6 hours to ensure that the threshold adapts to changes in system load.

[0062] When the topology entropy value exceeds a preset threshold, a migration path reconstruction instruction is generated. This instruction includes a list of path identifiers that must be prioritized for reconstruction and corresponding bandwidth constraints. The path identifier list is generated by filtering paths whose topology entropy values ​​exceed the threshold, excluding paths reconstructed within the last 30 minutes. The bandwidth constraints are dynamically set based on the current network bandwidth utilization. For example, if the overall bandwidth utilization exceeds 80%, the reconstruction bandwidth for a single path is limited to 5% of the total bandwidth; if the utilization is below 80%, the limit is relaxed to 10%. Bandwidth utilization is obtained in real-time through monitoring tools, which periodically read port traffic data from network switches.

[0063] The specific implementation method for analyzing the nonlinear resonance correlation strength of hardware performance degradation rates among different storage nodes and determining resonance node groups is as follows. The write lifetime degradation rate of the solid-state drives (SSDs) on each storage node is extracted as a performance degradation indicator. This rate is obtained by parsing the logs of the SSD's self-monitoring analysis and reporting technology tool. The logs record the change in the remaining write cycles of the SSD over time. The degradation rate is calculated as the difference between the current remaining write cycles and the remaining write cycles in the previous hour, divided by the time interval, which is fixed at 1 hour. The abnormal increase in the seek time of the mechanical hard drives (HDDs) on each storage node is also extracted as a performance degradation indicator. This abnormal increase is obtained by monitoring the seek operation time of the HDDs. The seek operation time is recorded by the hard drive firmware and read through the operating system interface. The abnormal increase is calculated as the difference between the current seek time and the historical average seek time. The historical average seek time is based on statistical analysis of the seek time data of the same HDD over the past 24 hours.

[0064] A covariance matrix is ​​constructed to represent the hardware performance degradation rate among different nodes. This covariance matrix quantifies the correlation between the performance degradation trends of different nodes. The covariance matrix is ​​constructed using a covariance matrix construction algorithm. This algorithm treats the SSD write lifespan degradation rate and the HDD seek time anomaly increment as two-dimensional vectors for each node, and calculates the covariance value between these vectors. The covariance value represents the linear correlation between the performance degradation trends of two nodes; a covariance value greater than 0 indicates a positive correlation, and a value less than 0 indicates a negative correlation. The number of rows and columns in the covariance matrix equals the total number of nodes, and the matrix elements are the covariance values ​​of corresponding node pairs.

[0065] Principal component analysis (PCA) is used to extract the largest eigenvalue of the covariance matrix as the nonlinear resonance correlation strength. PCA is a well-known statistical method used for dimensionality reduction and feature extraction. PCA decomposes the covariance matrix into eigenvalues, resulting in eigenvalues ​​and their corresponding eigenvectors sorted by size. The largest eigenvalue is the first principal component's variance contribution to the covariance matrix, representing the overall correlation strength based on the performance decay trend of different nodes. The component of the eigenvector corresponding to the largest eigenvalue represents the weight of each node in the overall correlation; a larger absolute weight indicates a higher contribution of that node to the overall correlation strength.

[0066] The nonlinear resonance correlation strength is compared with a critical threshold, which is dynamically set based on the statistical distribution characteristics of the covariance matrix eigenvalues. These statistical distribution characteristics are derived by analyzing a historical covariance matrix eigenvalue dataset, which contains the eigenvalues ​​generated hourly over the past 7 days. The critical threshold is set to the 95th quantile of this historical dataset. For example, if the largest eigenvalue in the historical dataset ranges from 0.5 to 2.0, and the 95th quantile is 1.9, then the critical threshold is set to 1.9. This dynamic setting process is performed every 6 hours to ensure the threshold adapts to changes in system load.

[0067] When the nonlinear resonance correlation strength exceeds a critical threshold, nodes with eigenvector weights in the covariance matrix exceeding a preset proportion are marked as resonance node groups. The preset proportion of eigenvector weights is set according to business needs. For example, for highly sensitive business scenarios, the preset proportion is set to 10%, meaning only the top 10% of nodes by weight are marked; for ordinary business scenarios, the preset proportion is relaxed to 20%. Weight ranking is achieved by sorting the absolute values ​​of each component of the eigenvector in descending order. Nodes marked within the last hour are excluded during marking to avoid duplicate processing.

[0068] The specific implementation method for generating dynamic node allocation weights to suppress oscillations based on nonlinear resonance correlation strength, resonance node groups, and path reconstruction instructions is as follows: The nonlinear resonance correlation strength is mapped to a suppression coefficient using a normalization function. The normalization function employs a linear scaling method to map the original value of the nonlinear resonance correlation strength to the interval between 0 and 1, ensuring a positive correlation between the suppression coefficient and the nonlinear resonance correlation strength. For example, if the historical maximum value of the nonlinear resonance correlation strength is 5.0 and the current value is 3.0, then the suppression coefficient is calculated as 3.0 divided by 5.0, resulting in 0.6. The maximum and minimum values ​​of the normalization function are dynamically adjusted based on the statistical values ​​of the nonlinear resonance correlation strength of the same storage node group over the past 24 hours, with the maximum value being 90% of the historical peak value and the minimum value being 50% of the historical average value.

[0069] A path reconstruction factor is generated based on the bandwidth occupancy constraints in the path reconstruction instruction. The path reconstruction factor is the product of the reciprocal of the topology entropy value of the path requiring priority reconstruction and the remaining available bandwidth. The topology entropy value of the path requiring priority reconstruction is obtained from the calculation results of the preceding steps. A higher topology entropy value indicates a greater degree of path disorder, and its reciprocal is smaller, resulting in a lower path reconstruction factor. The remaining available bandwidth is obtained in real-time by monitoring the port traffic data of the network switches. The monitoring tool collects bandwidth utilization data every 30 seconds, and the remaining available bandwidth is calculated as the total port bandwidth minus the currently used bandwidth. For example, if the topology entropy value of a path is 2.0 and the remaining available bandwidth is 200 Mbps, then the path reconstruction factor is 1 / 2.0 multiplied by 200, resulting in 100.

[0070] Dynamic node weights are generated based on a linear combination of suppression coefficients, path reconstruction factors, and real-time load status. The weight coefficients of the linear combination are optimized using gradient descent based on historical resource allocation error data. Real-time load status includes the remaining storage space and input / output pressure index of storage nodes. The remaining storage space is obtained through the storage management interface, and the input / output pressure index is calculated as the ratio of queue depth to a preset threshold.

[0071] The expression for the linear combination is: Node dynamic weight allocation = Weight A × Suppression coefficient + Weight B × Path reconstruction factor + Weight C × Remaining storage space + Weight D × Input / output pressure index. The initial values ​​of weights A, B, C, and D are all set to 0.25. Iterative optimization is achieved using gradient descent combined with historical resource allocation error data. The historical resource allocation error data is the weighted average of the number of migration oscillations and hardware performance degradation rates caused by unreasonable weight allocation over the past 7 days.

[0072] The gradient descent optimization process is achieved by minimizing the loss function, which is defined as the mean square error between the actual oscillation frequency and the expected frequency corresponding to the dynamically assigned weights of nodes. In each optimization iteration, the weight values ​​are adjusted based on the partial derivative of the loss function with respect to the weight coefficients, with an adjustment step size of 0.01. The maximum number of iterations is 1000 or until the rate of change of the loss function is less than 0.001. For example, if the loss function value decreases by less than 0.001 after a certain iteration, the optimization is terminated and the current weight coefficients are used. The optimized weight coefficients are stored in a configuration file, which is reloaded every 6 hours to adapt to changes in system load.

[0073] The specific implementation method for extracting prediction error data caused by sudden changes in hardware performance and path chaos in historical resource allocation, and generating error compensation coefficients by combining node dynamic allocation weights, is as follows: The number of data migration errors caused by a sudden drop in the write lifespan of the solid-state drive (SSD) is extracted from the resource allocation log. The number of data migration errors is the total number of data block migration failures or retries caused by a sudden decrease in the remaining write cycles of the SSD. Log entries are obtained by parsing event records marked "migration failed" or "retry" in the storage system operation log. Event records include timestamps, path identifiers, and error types. The number of invalid migration operations caused by path chaos is extracted from the resource allocation log. The number of invalid migration operations is the number of operations where data blocks are migrated and then re-migrated due to excessively high path topology entropy. Invalid migration operations are statistically determined by comparing the timestamps and path identifiers of the migration log and the re-migrating log. The time window matching tolerance is set to 1 minute.

[0074] The average of the number of data migration errors and invalid migration operations within a sliding time window is calculated and used as the standardized historical prediction error data. The sliding time window length is set to 1 hour, the sliding step size is set to 10 minutes, and the average is calculated as the arithmetic mean of the number of data migration errors and invalid migration operations within the window. For example, if there are 5 data migration errors and 3 invalid migration operations within the time window from 10:00 to 11:00, the standardized historical prediction error data will be (5+3) / 2 = 4. The start time of the sliding time window is aligned with the peak load period of the storage system. The peak load period is determined by analyzing the resource allocation logs of the past 7 days. For example, if the logs show that the peak period for migration operations is from 10:00 to 12:00 every day, then the window start time is set to 10:00.

[0075] The standardized historical prediction error data and the dynamically assigned node weights are input into the error compensation function to generate error compensation coefficients. The error compensation function is a nonlinear mapping relationship based on the hyperbolic tangent function, which compresses the input values ​​to the range of -1 to 1. By adjusting the function parameters, it is ensured that the error compensation coefficients monotonically decrease with the product of the historical prediction error data and the dynamically assigned node weights.

[0076] The error compensation coefficient is calculated using a hyperbolic tangent function. Specifically, the product of standardized historical prediction error data and dynamically assigned node weights is input into the function. The function output is compressed to the range of -1 to 1, and the output value monotonically decreases as the product value increases. The parameters of the hyperbolic tangent function include an attenuation factor and the base of the natural logarithm. The attenuation factor is dynamically adjusted based on the fluctuation range of the historical prediction error data. The fluctuation range is quantified by calculating the standard deviation of the historical prediction error data. A larger standard deviation indicates more severe data fluctuation, in which case the attenuation factor is smaller to reduce the attenuation rate of the compensation coefficient; conversely, a smaller standard deviation results in a larger attenuation factor to accelerate attenuation. The adjustment rule for the attenuation factor is that it is inversely proportional to the standard deviation, and its value is limited to between 0.1 and 1.0 to ensure the stability of the function output.

[0077] The application logic of the error compensation coefficient is as follows: when the product of the standardized historical prediction error data and the dynamically allocated weight of the node increases, the error compensation coefficient approaches -1, triggering the system to automatically reduce the task allocation priority of the current node; when the product decreases, the error compensation coefficient approaches 1, triggering the system to increase the node priority. For example, if the standardized historical prediction error data is 4 times and the dynamically allocated weight of the node is 0.6, the product of the two is 2.4. Combined with the attenuation factor of 0.5, the calculated error compensation coefficient is approximately -0.4, corresponding to a 40% reduction in node priority. The dynamic adjustment of the attenuation factor is synchronized with the step size of the sliding time window. Every 10 minutes, the attenuation factor is recalculated and updated based on the latest historical error data to ensure that the compensation coefficient matches the real-time error fluctuation trend. During the update process, the value of the attenuation factor is inversely proportional to the standard deviation of the historical error data. When the standard deviation increases, the attenuation factor decreases to smooth the change in the compensation coefficient; when the standard deviation decreases, the attenuation factor increases to accelerate the response.

[0078] The specific implementation of performing cross-storage node data migration operations based on the error compensation coefficient and limiting the allocation of high-frequency tasks to resonant node groups is as follows. The task allocation priority of each storage node is adjusted according to the error compensation coefficient. The implementation method where the task allocation priority is negatively correlated with the error compensation coefficient is as follows: the error compensation coefficient is multiplied by -1 and mapped to a priority range of 0 to 1. The lower limit of the priority range corresponds to the highest task allocation priority, and the upper limit corresponds to the lowest priority. For example, if the error compensation coefficient of a node is -0.4, then the task allocation priority is calculated as 0.4. The higher the priority value, the lower the qualification of the node to receive high-frequency tasks. The priority adjustment result is synchronized to the task scheduler in real time. The task scheduler allocates high-frequency tasks according to the priority value from low to high, ensuring that the proportion of high-frequency task allocation to low-priority nodes is suppressed.

[0079] A high-frequency task allocation ratio limit is imposed on the resonant node group. This limit is dynamically set based on the ratio of the node's dynamic allocation weight to its real-time load status. The node's dynamic allocation weight is obtained through a weight value generated in a previous step. The real-time load status includes the storage node's remaining storage space and input / output pressure index. The remaining storage space is queried in real-time through the storage management interface, and the input / output pressure index is calculated as the ratio of the queue depth to a preset threshold. The formula for calculating the high-frequency task allocation ratio limit is: Limit Ratio = Node Dynamic Allocation Weight / (Real-time Load Status + 1), where "+1" is used to prevent the denominator from being zero. For example, if the node's dynamic allocation weight is 0.6 and its real-time load status is 0.8, then the limit ratio = 0.6 / (0.8 + 1) = 0.33, indicating that the node's high-frequency task allocation ratio must not exceed 33%. The limit ratio is enforced by the task scheduler's quota management module. The quota management module periodically pulls the latest load data from the node monitoring service and updates the limit ratio every 5 minutes.

[0080] Based on the path identifier list in the migration path reconstruction instruction, data blocks stored in paths requiring priority reconstruction are migrated first. The data block migration process follows bandwidth occupancy constraints and updates the topology entropy value in real time. The identifier list of paths requiring priority reconstruction is obtained from the path identifier list generated in the previous steps. The bandwidth occupancy constraint stipulates that the bandwidth occupied by the migration operation of a single path must not exceed a preset percentage of the total available bandwidth. The preset percentage is dynamically adjusted according to the current network bandwidth utilization: if the bandwidth utilization is higher than 80%, the preset percentage is set to 5%; if it is lower than 80%, the preset percentage is relaxed to 10%. For example, if the total available bandwidth is 1000 Mbps and the current utilization is 85%, then the maximum migration bandwidth for path P1 is 1000 × 5% = 50 Mbps. After the data block migration is completed, the topology entropy value of the path is updated in real time. The update method is to recalculate the inter-node connection relationships and migration frequency of the path, and synchronize the new entropy value to the path management database to ensure that subsequent resource allocation decisions are based on the latest topology status.

[0081] This embodiment systematically solves the resource allocation oscillation problem caused by sudden changes in hardware performance and path chaos in traditional technologies through dynamic error compensation and multi-dimensional resource regulation mechanisms. Based on the dynamic mapping of task allocation priorities using error compensation coefficients, a negative correlation design ensures that high-error nodes are automatically downweighted, preventing inefficient task allocation from exacerbating oscillations. Compared to traditional static threshold strategies, priority values ​​are strictly bound to real-time error status, preventing a vicious cycle of error accumulation. The ratio constraint between the dynamic weight of inbound nodes and real-time load considers hardware performance degradation (dynamic weight) and instantaneous load pressure (real-time load) in tandem, preventing overloaded nodes from further deteriorating performance due to high-frequency tasks. For example, the proportion of tasks for nodes with low dynamic weights and high loads is significantly reduced, decreasing the risk of collective hardware degradation. Combining the dual constraints of path entropy and bandwidth usage, highly chaotic paths are prioritized for reconstruction, and migration bandwidth is strictly limited. This optimizes topology order through entropy value updates in a closed loop while preventing migration operations from crowding out business traffic. Compared to traditional random migration or pure load-driven strategies, this simultaneously improves path stability and business continuity.

[0082] Parameters such as error compensation coefficient, attenuation factor, and limit ratio are dynamically adjusted based on historical and real-time data, replacing the insufficient adaptability caused by fixed parameters. Multi-dimensional indicators such as hardware performance, path status, and network load are cross-validated to overcome the limitations of single-indicator decision-making. Core parameters such as priority, limit ratio, and entropy are updated on a minute-by-minute basis to ensure that response speed matches the system disturbance frequency. Compared to existing technologies, this combined design of dynamic weight attenuation, closed-loop path entropy updates, and elastic bandwidth constraints reduces resource allocation oscillation frequency, decreases hardware lifespan loss, and reduces invalid migration operations.

[0083] By employing a synergistic mechanism of topological entropy monitoring and nonlinear resonance analysis, a dual suppression barrier is constructed to address the two core issues of path disorder and hardware group degradation. Topological entropy monitoring dynamically calculates the path structure disorder based on the connection relationships between nodes and migration frequency. When the entropy value exceeds the limit, a path reconstruction command is triggered to optimize the data distribution topology in real time, preventing local oscillations caused by path disorder. Nonlinear resonance analysis quantifies the group correlation strength of hardware performance degradation through the covariance matrix, identifies resonant node groups, and limits their high-frequency task load to prevent single-node performance mutations from becoming hotspots of group degradation. The two mechanisms are jointly regulated through a dynamic weight generation module: the path reconstruction factor suppresses the increase of topological entropy, and the resonance suppression coefficient alleviates hardware resonance, ultimately forming a two-way negative feedback control of path order and hardware state, suppressing local oscillations at the nascent stage and preventing them from spreading globally through topological connections or hardware coupling.

[0084] All calculations involved in the embodiments are dimensionless numerical calculations, and the preset parameters and thresholds in the calculations are set by those skilled in the art according to the actual situation.

[0085] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0086] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0087] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0088] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0090] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0091] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An artificial intelligence-based computing platform management system, characterized in that, Includes the following modules: Real-time acquisition module: Acquires hardware performance indicators, real-time load status and data access paths of each storage node in the distributed storage system in real time; Entropy determination module: Calculates the topology entropy value of the data access path. When the topology entropy value exceeds the preset threshold, it is determined to be a chaotic migration path and triggers a path reconstruction command. Resonance Analysis Module: Based on hardware performance indicators, analyzes the nonlinear resonance correlation strength of the hardware performance degradation rate between different storage nodes. When the nonlinear resonance correlation strength exceeds a critical threshold, the resonance node group is determined. Weight generation module: Based on the nonlinear resonance correlation strength, resonance node group and path reconstruction instructions, it generates dynamic weights for nodes to suppress oscillations through a dynamic feature weighting model; Error compensation module: Extracts prediction error data caused by sudden changes in hardware performance and path chaos in historical resource allocation, and generates error compensation coefficients by combining the dynamic weight allocation of nodes. Migration Execution Module: Performs data migration operations across storage nodes based on the error compensation coefficient. The data migration operation restricts the allocation of high-frequency tasks in the resonant node group and prioritizes the reconstruction of disordered paths. Based on the nonlinear resonance correlation strength, resonance node group, and path reconstruction instructions, a dynamic feature weighting model is used to generate dynamic node weights to suppress oscillations, including: The nonlinear resonance correlation strength is mapped to a suppression coefficient through a normalization function. The normalization function ensures that the suppression coefficient is positively correlated with the nonlinear resonance correlation strength. The path reconstruction factor is generated based on the bandwidth occupancy constraints in the path reconstruction instruction. The path reconstruction factor is the product of the reciprocal of the topology entropy value of the path that needs to be reconstructed first and the remaining available bandwidth. Dynamic weight allocation for nodes is generated based on a linear combination of inhibition coefficient, path reconstruction factor, and real-time load status.

2. The AI-based computing platform management system according to claim 1, characterized in that, Real-time acquisition of hardware performance metrics, real-time load status, and data access paths for each storage node in the distributed storage system, including: The system collects the read / write speed, latency, and storage media lifespan of storage nodes as hardware performance indicators. The storage media lifespan is quantified by the remaining write cycles of solid-state drives and the cumulative runtime of hard disk drives. The system also collects the real-time load status of storage nodes, including remaining storage space and input / output pressure. Finally, the system collects the inter-node connectivity and data migration frequency of the data access path.

3. The AI-based computing platform management system according to claim 1, characterized in that, Calculate the topology entropy value of the data access path. When the topology entropy value exceeds a preset threshold, it is determined to be a chaotic migration path state and a path reconstruction instruction is triggered, including: Based on the adjacency matrix of inter-node connections and the time series of data migration frequencies, the topological entropy value is calculated using the Shannon entropy formula. The Shannon entropy formula states that the topological entropy value is equal to the weighted sum of the information entropy of the probability distribution of inter-node connections and the coefficient of variation of the data migration frequency. The topology entropy value is compared with a preset threshold, which is dynamically adjusted based on the upper bound of the historical topology entropy value statistical distribution. When the topology entropy value exceeds a preset threshold, a migration path reconstruction instruction is generated. The migration path reconstruction instruction includes a list of path identifiers that need to be reconstructed first and the corresponding bandwidth occupancy constraints.

4. The AI-based computing platform management system according to claim 3, characterized in that, The coefficient of variation of data migration frequency is calculated by dividing the standard deviation of the time series of data migration frequency by the mean.

5. The AI-based computing platform management system according to claim 1, characterized in that, The nonlinear resonance correlation strength of the hardware performance degradation rate among different storage nodes is analyzed based on hardware performance indicators. When the nonlinear resonance correlation strength exceeds a critical threshold, a resonant node group is determined, including: Extract the write life degradation rate of solid-state drives and the abnormal increase in seek time of mechanical hard drives from each storage node as time-series data of hardware performance degradation rate. Construct a covariance matrix of hardware performance degradation rate among different nodes, and extract the maximum eigenvalue of the covariance matrix as the nonlinear resonance correlation strength using principal component analysis. The nonlinear resonance correlation strength is compared with a critical threshold, which is dynamically set based on the statistical distribution characteristics of the eigenvalues ​​of the covariance matrix. When the nonlinear resonance correlation strength exceeds the critical threshold, nodes whose eigenvector weights in the covariance matrix exceed a preset ratio are marked as resonance node groups.

6. The AI-based computing platform management system according to claim 1, characterized in that, The weighting coefficients of the linear combination are optimized using gradient descent based on historical resource allocation error data.

7. The AI-based computing platform management system according to claim 1, characterized in that, Extract prediction error data caused by sudden changes in hardware performance and path chaos in historical resource allocation, and generate error compensation coefficients by combining them with dynamically allocated node weights, including: Extract the number of data migration errors caused by a sudden drop in the write lifespan of the solid-state drive from the resource allocation log, and the number of invalid migration operations caused by path chaos as historical prediction error data; The mean of the sliding time window for the number of data migration errors and the number of invalid migration operations is calculated and used as the standardized historical prediction error data. The standardized historical prediction error data and the dynamically assigned node weights are input into the error compensation function to generate error compensation coefficients.

8. The AI-based computing platform management system according to claim 7, characterized in that, The error compensation function is a nonlinear mapping relationship based on the hyperbolic tangent function, ensuring that the error compensation coefficient decreases monotonically with the product of historical prediction error data and the dynamically allocated weights of nodes.

9. The artificial intelligence-based computing platform management system according to claim 1, characterized in that, Data migration operations across storage nodes are performed based on error compensation coefficients. These operations restrict the allocation of high-frequency tasks to the resonant node group and prioritize the reconstruction of disordered paths, including: The task allocation priority of each storage node is adjusted according to the error compensation coefficient, and the task allocation priority is negatively correlated with the error compensation coefficient. A high-frequency task allocation ratio limit is imposed on the resonant node group. The high-frequency task allocation ratio limit is dynamically set according to the ratio of the node's dynamic allocation weight to its real-time load status. Based on the path identifier list in the migration path reconstruction instruction, data blocks stored in the paths that need to be reconstructed should be migrated first. The data block migration process follows bandwidth occupancy constraints and updates the topology entropy value in real time.

Images