Directory binding method, device, equipment and product based on distributed system CephFS

Abstract

The application discloses a directory binding method and device based on a distributed system CephFS, equipment and products, relates to the technical field of load balancing, and the method is applied to CephFS, and the method comprises the following steps: collecting real-time load data of a metadata server MDS set, and generating a load feature vector according to the load data; collecting historical load data of the MDS set, constructing a historical load portrait according to the historical load data, and performing load prediction on the historical load data through a combined prediction model to obtain predicted load; when it is detected that there is a directory adjustment request, calculating the load adaptation degree of the MDS set according to the directory adjustment request, the load feature vector, the historical load portrait and the predicted load; and binding a target directory to a target MDS with the highest load adaptation degree in the MDS set according to the directory adjustment request. Therefore, the efficiency of load adjustment is improved.

Description

Technical Field

[0001] This application relates to the field of load balancing technology, and in particular to a directory binding method, apparatus, device, and product based on the distributed system CephFS. Background Technology

[0002] CephFS's multi-MDS cluster architecture improves concurrent access performance through metadata sharding. The binding and allocation of directories and MDSs is the core factor determining cluster efficiency. Existing technologies mainly include three types: first, the static Pin mechanism, where operations personnel manually assign fixed MDSs to directories; second, the temporary Pin mechanism, which uses random or round-robin strategies to allocate target MDSs when directories are created; and finally, the traditional metadata load balancing method, which monitors the utilization of basic resources such as CPU and memory, and triggers "post-migration" when the load unevenness exceeds a threshold, migrating hot directories from high-load MDSs to low-load MDSs. In addition, existing load assessment mainly relies on threshold judgments of resource consumption indicators.

[0003] However, all of the above-mentioned existing technologies have significant drawbacks. Static Pin mechanisms rely entirely on human experience and cannot adapt to the dynamic creation needs of massive directories in cloud-native scenarios. Frequent manual configuration is not only impractical but also prone to errors. Temporary Pin mechanisms, with their random or distributed strategies, are essentially "indiscriminate allocations," ignoring the real load differences between different MDSs. This can easily lead to an unbalanced situation where "busy ones get busier and idle ones get idler." Traditional "post-migration" models suffer from a combination of drawbacks, including IO interruptions caused by migration locks (which can reach hundreds of milliseconds to seconds), huge overhead from full quantum tree metadata replication, and performance losses due to response lag that have already occurred. These drawbacks severely impact business continuity and cluster efficiency. At the same time, existing load assessments only focus on basic resources such as CPU and memory, completely ignoring metadata operation characteristics (such as lock contention intensity and log writing pressure) and load patterns over time (such as periodic peaks). This results in low accuracy in allocation decisions and makes it difficult to fundamentally solve the load balancing problem of multi-MDS clusters.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a directory binding method, device, equipment and product based on the distributed system CephFS, which aims to solve the technical problem that the directory allocation in the load management of the MDS set in the existing CephFS system lacks multi-dimensional load perception and forward prediction, making it difficult to achieve efficient dynamic load balancing.

[0006] To achieve the above objectives, this application proposes a directory binding method based on the distributed system CephFS. This directory binding method, applied to CephFS, includes: Collect real-time load data from the metadata server MDS set, and generate a load feature vector based on the load data; Historical load data of the MDS set is collected, a historical load profile is constructed based on the historical load data, and load prediction is performed on the historical load data through a combined prediction model to obtain the predicted load. When a directory adjustment request is detected, the load fit of the MDS set is calculated based on the directory adjustment request using the load feature vector, the historical load profile, and the predicted load. Based on the directory adjustment request, the target directory is bound to the target MDS with the highest load adaptability in the MDS collection.

[0007] In one embodiment, the step of constructing a historical load profile based on the historical load data includes: The historical load data is stored according to a sliding window of preset time length to obtain window data; Perform time-domain statistical analysis on the window data to obtain time-domain characteristics; Periodic characteristics were obtained by performing periodic analysis on the window data using spectral analysis. The trend characteristics of the window data are determined by difference analysis and regression analysis. The historical load profile corresponding to the MDS set is constructed based on the time-domain features, the periodic features, and the trend features.

[0008] In one embodiment, the combined prediction model includes a first prediction model and a second prediction model. The step of using the combined prediction model to predict the load from the historical load data to obtain the predicted load includes: The historical load data is decomposed into seasonal, trend, and random components. The trend component is input into the first prediction model for prediction to obtain the trend prediction value; The random component is input into the second prediction model for prediction to obtain a random predicted value; The predicted load is obtained by fusing the seasonal component, the trend prediction value, and the random prediction value.

[0009] In one embodiment, the step of calculating the load fit of the MDS set based on the directory adjustment request using the load feature vector, the historical load profile, and the predicted load includes: The directory adjustment request is parsed to determine the directory type and access mode of the target directory, and the directory characteristics are obtained based on the directory type and access mode. Calculate the real-time load score of several MDSs in the MDS set based on the load feature vector; Calculate the historical fit score between the directory features and the several MDSs based on the historical load profile; Calculate the predicted load score of the plurality of MDSs based on the predicted load; The matching score between the directory access pattern and the several MDSs is calculated based on the directory features and the hardware characteristics of the several MDSs. The load fit is obtained by weighted summing of the real-time load score, the historical fit score, the predicted load score, and the pattern matching score.

[0010] In one embodiment, the step of binding the target directory to the target MDS with the highest load balancing in the MDS cluster according to the directory adjustment request includes: The MDS set is filtered according to the load adaptability to obtain the target MDS; Based on the directory adjustment request, the target directory is bound to the attribute identifier of the target MDS, and the metadata access of the target directory is migrated to the target MDS to obtain the binding result.

[0011] In one embodiment, after the step of binding the target directory to the target MDS with the highest load balancing in the MDS centralization according to the directory adjustment request, the method further includes: When the migration triggering conditions are met, the metadata of the target directory is copied to the alternative MDS for preheating to obtain the preheating result; Update the binding attribute identifier of the target directory to the alternative MDS, and notify the client to switch the access target to obtain the switching result; Verify the directory access status of the candidate MDS and obtain the verification results; If the verification result is a verification failure, the binding attribute identifier is restored to the original MDS, and the client is notified.

[0012] In one embodiment, after the step of binding the target directory to the target MDS with the highest load balancing in the MDS centralization according to the directory adjustment request, the method further includes: The directory access latency, binding decision accuracy, and migration success rate of the MDS set were collected. The server monitoring data is obtained by summarizing the directory access latency, binding decision accuracy, and migration success rate. When the preset adjustment period is reached, the weight of the load adaptability index is adjusted according to the server monitoring data.

[0013] Furthermore, to achieve the above objectives, this application also proposes a directory binding device based on the distributed system CephFS, wherein the directory binding device based on the distributed system CephFS is applied to CephFS, and the directory binding device based on the distributed system CephFS includes: The acquisition module is used to acquire real-time load data of the metadata server MDS set and generate a load feature vector based on the load data. The construction module is used to collect historical load data of the MDS set, construct a historical load profile based on the historical load data, and perform load prediction on the historical load data through a combined prediction model to obtain the predicted load. The calculation module is used to calculate the load adaptability of the MDS set based on the load feature vector, the historical load profile and the predicted load when a directory adjustment request is detected. The migration module is used to bind the target directory to the target MDS with the highest load adaptability in the MDS collection according to the directory adjustment request.

[0014] Furthermore, to achieve the above objectives, this application also proposes a directory binding device based on the distributed system CephFS, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the directory binding method based on the distributed system CephFS as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the directory binding method based on the distributed system CephFS as described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the directory binding method based on the distributed system CephFS as described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: This application proposes a directory binding method, apparatus, device, and product based on the distributed system CephFS. The method involves: collecting real-time load data from a metadata server (MDS) set; generating a load feature vector based on the load data; collecting historical load data from the MDS set; constructing a historical load profile based on the historical load data; and using a combined prediction model to predict the load on the historical load data to obtain a predicted load. Upon detecting a directory adjustment request, the method calculates the load fit of the MDS set based on the directory adjustment request, using the load feature vector, the historical load profile, and the predicted load. Finally, the method binds the target directory to the target MDS set with the highest load fit based on the directory adjustment request. Therefore, by collecting multi-dimensional load data to generate load feature vectors, a fine perception of the current state of the MDS is achieved. Secondly, historical load profiles are constructed and predicted loads are obtained using a combined prediction model, giving the system forward-looking prediction capabilities and avoiding response lags in post-migration. On this basis, the load fit is calculated by integrating the current load, historical profiles, and predicted loads to achieve multi-dimensional fusion evaluation. Finally, the MDS with the highest fit is selected through intelligent pin decision-making, and the binding relationship between the directory and the MDS is adjusted to achieve intelligent dynamic allocation of directory metadata service nodes, rather than simple directory migration. At the same time, when dynamically adjusting the directory binding relationship at runtime, the risk of IO interruption is minimized by using the obtained load feature vectors, historical load profiles, and predicted loads, ensuring business continuity. This solves the problem in the existing CephFS system's MDS set load management where directory allocation lacks multi-dimensional load perception and forward-looking prediction, making it difficult to achieve efficient dynamic load balancing, and improves the efficiency of load adjustment. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the directory binding method based on the distributed system CephFS in this application. Figure 2 This diagram illustrates the combined prediction model used in the directory binding method based on the distributed system CephFS in this application. Figure 3This is a flowchart illustrating Embodiment 2 of the directory binding method based on the distributed system CephFS in this application; Figure 4 This is a schematic diagram illustrating the selection of the target metadata service area involved in the directory binding method based on the distributed system CephFS in this application; Figure 5 This diagram illustrates the switch verification involved in the directory binding method based on the distributed system CephFS in this application. Figure 6 A simplified flowchart illustrating the directory binding method based on the distributed system CephFS provided in Embodiment 2 of this application; Figure 7 This is a schematic diagram of the module structure of the directory binding device based on the distributed system CephFS in an embodiment of this application; Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the directory binding method based on the distributed system CephFS in the embodiments of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of this application embodiment is as follows: Historical load data is stored according to a sliding window of a preset time length to obtain window data; time-domain statistical analysis is performed on the window data to obtain time-domain features; periodic analysis is performed on the window data using spectral analysis to obtain periodic features; trend judgment is performed on the window data using difference analysis and regression analysis to obtain trend features; a historical load profile corresponding to the MDS set is constructed based on the time-domain features, the periodic features, and the trend features. The historical load data is decomposed into seasonal components, trend components, and random components; the trend components are input into the first prediction model for prediction to obtain trend prediction values; the random components are input into the second prediction model for prediction to obtain random prediction values; the seasonal components, the trend prediction values, and the random prediction values ​​are fused to obtain the predicted load. The process involves parsing the directory adjustment request to determine the directory type and access mode of the target directory, and obtaining directory features based on these characteristics. Real-time load scores are calculated for several MDSs in the MDS set based on the load feature vector. Historical fit scores between the directory features and the several MDSs are calculated based on the historical load profile. Predicted load scores are calculated for the several MDSs based on the predicted load. Pattern matching scores between the directory access mode and the several MDSs are calculated based on the directory features and the hardware characteristics of the several MDSs. A weighted sum of the real-time load score, the historical fit score, the predicted load score, and the pattern matching score is then obtained to determine the load fit. The MDS set is then filtered based on the load fit to obtain the target MDS. Finally, based on the directory adjustment request, the target directory is bound to the attribute identifier of the target MDS, and the metadata access of the target directory is migrated to the target MDS, resulting in a binding result. When the migration trigger condition is met, the metadata of the target directory is copied to a candidate MDS for preheating, and the preheating result is obtained; the binding attribute identifier of the target directory is updated to the candidate MDS, and the client is notified to switch the access target, and the switching result is obtained; the directory access status of the candidate MDS is verified, and the verification result is obtained; if the verification result is a verification failure, the binding attribute identifier is restored to the original MDS, and the client is notified. The directory access latency, binding decision accuracy, and migration success rate of the MDS set are collected; the directory access latency, binding decision accuracy, and migration success rate are summarized to obtain server monitoring data; when the preset adjustment cycle is reached, the weight of the load adaptability indicator is adjusted according to the server monitoring data.This invention addresses the problem in existing CephFS systems where MDS cluster load management lacks multi-dimensional load awareness and forward-looking prediction in directory allocation, hindering efficient dynamic load balancing. It achieves load adjustment and improves efficiency. Based on this invention, it addresses the issue that existing load assessments only focus on basic resources like CPU and memory, completely ignoring metadata operation characteristics (such as lock contention intensity and log write pressure) and time-based load patterns (such as periodic peaks). This leads to low accuracy in allocation decisions and an inability to fundamentally solve the load balancing problem in multi-MDS clusters, resulting in low efficiency. Therefore, this invention designs a directory binding method based on the distributed system CephFS. The effectiveness of this CephFS-based directory binding method is verified during load adjustment, and the efficiency of load adjustment is significantly improved.

[0025] In this embodiment, for ease of description, the following description will focus on the directory binding device based on the distributed system CephFS.

[0026] Due to the limitations of existing directory and MDS allocation mechanisms, the efficiency of load balancing needs improvement. One issue is manual dependency: static pin mechanisms rely entirely on manual configuration by operations personnel, which cannot meet the dynamic creation needs of massive directories in cloud-native scenarios. Another issue is indiscriminate allocation: the random or round-robin strategies of temporary pin mechanisms ignore the real load differences between different MDSs, easily leading to an unbalanced situation where "busy ones get busier and idle ones get idler." Furthermore, there is the issue of response lag: traditional "post-migration" models suffer from IO interruptions caused by migration locks, huge overhead from full quantum tree metadata replication, and performance losses due to response lag. Therefore, in current CephFS multi-MDS clusters, domain adaptability also faces challenges. Existing load assessments only focus on basic resources such as CPU and memory, ignoring metadata operation characteristics and load patterns over time, resulting in low accuracy in allocation decisions and difficulty in achieving efficient dynamic load balancing.

[0027] This application provides a solution that generates a load feature vector by collecting multi-dimensional load data, enabling a fine-grained perception of the current state of the MDS. Secondly, it constructs a historical load profile and uses a combined prediction model to obtain predicted load, giving the system forward-looking predictive capabilities and avoiding lag in post-migration response. Based on this, it calculates load fit by integrating current load, historical profile, and predicted load, achieving multi-dimensional fusion evaluation. Finally, it selects the MDS with the highest fit through intelligent pin decision-making, adjusts the binding relationship between the directory and the MDS, and achieves intelligent dynamic allocation of directory metadata service nodes, rather than simple directory migration. Simultaneously, when dynamically adjusting the directory binding relationship at runtime, it minimizes IO interruption risks by using the acquired load feature vector, historical load profile, and predicted load, ensuring business continuity. This solves the problem in existing CephFS systems where MDS load management lacks multi-dimensional load perception and forward-looking prediction, making it difficult to achieve efficient dynamic load balancing, thus improving the efficiency of load adjustment.

[0028] Based on this, this application provides a directory binding method based on the distributed system CephFS, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the directory binding method based on the distributed system CephFS in this application.

[0029] In this embodiment, the directory binding method based on the distributed system CephFS is applied to CephFS, including steps S01~S04: Step S01: Collect real-time load data from the metadata server MDS set, and generate a load feature vector based on the load data. Before the implementation of this embodiment, it should be clear that existing load assessment only focuses on basic resources such as CPU and memory, completely ignoring the characteristics of metadata operations (such as lock contention intensity and log writing pressure) and the load patterns in the time dimension (such as periodic peaks), resulting in low accuracy of allocation decisions and making it difficult to fundamentally solve the load balancing problem of multi-MDS clusters.

[0030] Therefore, in order to solve the above problems, this embodiment collects multi-dimensional load operation data of the metadata server MDS in real time, covering five major categories of core indicators: hardware resources, metadata operations, cache status, client interaction and cluster collaboration. The collected raw load data is processed by removing outliers, filling in missing data and standardizing and normalizing. After eliminating the differences in the dimensions of different indicators, a weighted fusion calculation is performed according to the preset indicator weights to finally generate a load feature vector that can accurately represent the real-time operating status of each metadata server.

[0031] Specifically, the five categories of load metrics and collection rules in this embodiment are as follows: Resource usage includes CPU utilization, memory utilization, and disk IO utilization, collected at a frequency of 1 second / time, with data sourced from the MDS node system monitoring interface; Metadata operation includes metadata IOPS, average access latency of various metadata requests, and the percentage of operation types such as read, write, create, and delete, collected at a frequency of 1 second / time, with data sourced from the CephFS perf dump counter; mdcache load status includes mdcache cache utilization and caps cache count, collected at a frequency of 500ms / time, with data sourced from the MDSdaemonperf module; MDS and client interaction includes the number of active sessions and the number of newly created sessions, collected at a frequency of 1 second / time, with data sourced from the MDSdaemonperf module; Cluster collaboration includes communication latency with Mon / OSD, metadata synchronization success rate, and cluster health status score, collected at a frequency of 2 seconds / time, with data sourced from the Ceph cluster monitoring interface.

[0032] Data preprocessing employs a three-level process. First, the 3σ criterion is used to remove extreme outliers from the collected data and filter out invalid data such as instantaneous high latency caused by network jitter. Second, missing data is filled in using linear interpolation to ensure the continuity of time-series data. Finally, a deviation standardization method is used to dynamically update the upper and lower limits based on the extreme values ​​of indicators over the past 7 days, mapping all indicators to the [0, 1] interval to eliminate the magnitude differences between different indicators. After standardization, the load is weighted and summed according to the following weights: resource occupancy (0.3), metadata operation (0.3), mdcache load (0.2), client interaction (0.1), and cluster collaboration (0.1) to obtain the current comprehensive load value of each MDS, which constitutes the load feature vector.

[0033] Step S02: Collect historical load data of the MDS set, construct a historical load profile based on the historical load data, and perform load prediction on the historical load data through a combined prediction model to obtain the predicted load. We continuously collect time-series historical load data from each metadata server within the cluster. We complete the sharding and organization of historical data through a fixed-size sliding window. We then mine time-domain statistical features, periodic fluctuation features, and trend change features from the organized time-series data. Combined with multi-dimensional feature data, we build a unique historical load profile that adapts to the operating rules of each metadata server.

[0034] Simultaneously, historical load time-series data is decomposed into components, and accurate prediction of different time-series components is achieved through the collaboration of dual models. The prediction results of multiple components are integrated to obtain the predicted load data of each metadata server for future periods, thereby realizing the advance prediction of load status.

[0035] Step S03: When a directory adjustment request is detected, the load fit of the MDS set is calculated based on the directory adjustment request using the load feature vector, the historical load profile, and the predicted load. The system monitors directory creation, migration, and other directory adjustment requests in the cluster in real time. Upon triggering a request, it parses the inherent attributes and operational characteristics of the target directory. Combining the real-time load status, historical operational patterns, and future load prediction results of each metadata server, it performs a comprehensive quantitative score from four dimensions: real-time load adaptability, historical operational matching, future load security, and access pattern fit. By weighted and fused multi-dimensional score results, it obtains the exclusive load adaptability of each metadata server for the target directory, accurately quantifying the degree of matching between the server and the directory to be adjusted.

[0036] Step S04: Bind the target directory to the target MDS with the highest load adaptability in the MDS set according to the directory adjustment request.

[0037] Based on the calculated load adaptability of each server, the optimal node is selected, and the metadata server with the best adaptability is selected as the target binding node. This embodiment is not a simple directory migration, but rather a precise selection of the MDS with the highest adaptability through three-dimensional load assessment (current load, historical profile, and predicted load), and adjustment of the binding relationship between the directory and the MDS (i.e., the intelligent Pin mechanism) to realize the intelligent dynamic allocation of directory metadata service nodes. The exclusive binding identifier (ceph.dir.pin) between the target directory and the target server is configured through cluster attribute configuration, all metadata access permissions corresponding to the directory are migrated synchronously, and the core metadata associated with the directory is actively cached and warmed up to complete the intelligent binding and load migration of the directory, thereby achieving dynamic load balancing of the cluster.

[0038] Specifically, step S02 above, the step of constructing a historical load profile based on the historical load data, includes: Step S021: Store the historical load data according to a sliding window of a preset time length to obtain window data; Step S022: Perform time-domain statistical analysis on the window data to obtain time-domain features; Step S023: Perform periodic analysis on the window data using the spectrum analysis method to obtain periodic characteristics; Step S024: The window data is analyzed for trends using differential analysis and regression analysis to obtain trend characteristics; Step S025: Construct a historical load profile corresponding to the MDS set based on the time-domain features, the periodic features, and the trend features.

[0039] In this embodiment, a 24-hour duration is selected as the statistical period for the sliding window, which can fully cover the daily load operation pattern of the server. A certain window sliding step size (e.g., 1 hour, 2 hours, etc.) is set to ensure the timeliness of load characteristics while effectively controlling data computation and storage overhead. The collected continuous time-series historical load data is segmented, classified, and stored according to the sliding window specification, forming a standardized window time-series dataset. The window data is stored using the InfluxDB time-series database, supporting fast querying by indicator type and time range. In addition to the three types of features—time domain, periodicity, and trend—abnormal features of the window data are extracted simultaneously. A 3σ criterion combined with moving average filtering is used to remove abnormal data points exceeding the confidence interval and smooth noise, resulting in a purified historical load sequence for subsequent prediction model training.

[0040] We conduct comprehensive time-domain statistical analysis on the window dataset to accurately calculate core statistical parameters such as the mean, variance, and peak frequency of the load data within the window. This allows us to characterize the baseline operating level, fluctuation range, and high-frequency load status of the server load, forming a complete time-domain feature system.

[0041] Fast Fourier Transform (FFT) spectrum analysis is employed to analyze window data in the frequency domain, extracting the fixed fluctuation period of the load data. This accurately identifies the periodic operating pattern of the server load, distinguishing between stable daily periods and peak periods, and extracting periodic characteristics. FFT is a mathematical transformation method that converts time-domain signals to the frequency domain for analysis. Its core principle is to discretely sample continuous time-series load data and then rapidly calculate the Discrete Fourier Transform (DFT) using a butterfly operation. This decomposes the waveform of the load changing over time into several sinusoidal components of different frequencies, thus visually presenting the energy distribution of the load data in the frequency domain. In this embodiment, by performing FFT on historical load time-series data, the dominant frequency component and its harmonics can be clearly identified in the frequency domain, such as the fundamental frequency corresponding to a 24-hour cycle and the second harmonic corresponding to a 12-hour cycle. This accurately locates the inherent periodic pattern of the server load, distinguishing between stable daily operating periods and peak business periods, providing accurate periodic characteristic basis for subsequent load prediction and intelligent decision-making.

[0042] By performing a first-order difference algorithm on continuous time-series load data to eliminate data trend interference, and combining this with a linear regression algorithm to fit and analyze the overall data trend, the system can accurately determine whether the server load is in a continuously rising, continuously falling, or stable fluctuating trend, thus completing trend feature extraction. The first-order difference algorithm is a mathematical transformation method for processing time-series data. Its core principle is to calculate the difference between load data at consecutive adjacent time points, i.e., to calculate the load change between the current moment and the previous moment. Through first-order difference operations, long-term trend components and seasonal fluctuations in the original time-series data can be effectively eliminated, transforming non-stationary sequences into stationary sequences and highlighting the instantaneous direction and magnitude of load changes. In this embodiment, by performing first-order difference on historical load time-series data, baseline offsets and periodic fluctuations of the load can be filtered out, while retaining short-term fluctuation characteristics. Then, by combining this with linear regression to fit and analyze the difference sequence, the system can accurately determine whether the server load is in a continuously rising, continuously falling, or stable fluctuating trend based on the magnitude and sign of the regression slope, providing a stable data foundation for subsequent trend feature extraction and load prediction.

[0043] Finally, by integrating the acquired time-domain statistical features, periodic fluctuation features, and trend features, and combining them with preset profile classification standards, each metadata server is divided into five exclusive historical load profiles: stable load, peak concentration, continuously rising load, continuously falling load, and drastic fluctuation load. This fully restores the long-term operating patterns and load characteristics of each server, providing accurate data support for subsequent directory matching decisions and load prediction. Among them, the stable load type corresponds to a load variance ≤ 0.1 and a trend slope in the range of [-0.05, 0.05]; the peak concentration type corresponds to a daily fixed-period load peak ≥ 0.8 and significant periodic characteristics; the continuously rising type corresponds to a trend slope > 0.05 and load increase for 3 consecutive hours; the continuously falling type corresponds to a trend slope < -0.05 and load decrease for 3 consecutive hours; and the drastic fluctuation type corresponds to a load variance ≥ 0.2 and no obvious periodic characteristics.

[0044] More specifically, the aforementioned combined prediction model includes a first prediction model and a second prediction model. Step S02, which involves using the combined prediction model to predict the historical load data and obtain the predicted load, includes: Step S026: Perform time-series decomposition on the historical load data to obtain seasonal components, trend components, and random components; Step S027: Input the trend component into the first prediction model for prediction to obtain the trend prediction value; Step S028: Input the random component into the second prediction model for prediction to obtain a random predicted value; Step S029: The seasonal component, the trend prediction value, and the random prediction value are fused to obtain the predicted load.

[0045] This implementation example Figure 2 As shown, a high-precision combined prediction system is constructed using the STL time series decomposition algorithm, the LSTM first prediction model, and the XGBoost second prediction model. The overall model architecture has a clear hierarchical structure and well-defined division of labor, which can adapt to the complex fluctuation characteristics of time series data under the load of metadata servers. Among them, STL (Seasonal and Trend decomposition using Loess) is a classic statistical method that robustly decomposes time series data into seasonal components, trend components, and random components. Its core principle is to separate components of different frequencies by repeatedly applying Loess smoothing (local weighted regression). The algorithm first estimates the trend component, subtracts the trend from the original sequence to obtain the detrended sequence, then segments the detrended sequence according to the period length, and performs local weighted regression smoothing on the data points of the same phase in each period to obtain the preliminary seasonal components. Finally, low-pass filtering is used to remove the residual trend in the seasonal components, and the seasonally adjusted sequence is calculated. Finally, Loess smoothing is applied again to the seasonally adjusted sequence to obtain the final trend component. The random component is obtained by subtracting the trend and seasonal components from the original sequence. The entire process is carried out through inner and outer loops until convergence. The inner loop is responsible for updating the trend and seasonal components, while the outer loop reduces the impact of outliers through robust weights, thereby obtaining a stable and noise-resistant decomposition result. In this embodiment, the seasonal period of STL decomposition is determined by the period extracted by the aforementioned FFT, with a default 24-hour daily period. The local smoothing window is set to 7 data points. The seasonal component directly reuses the load data of the same period in history, preserving the inherent cyclical operation pattern of the server load.

[0046] First, the cleaned and standardized historical load time-series data is decomposed into layers using the STL seasonal trend decomposition algorithm. This accurately separates seasonal components with fixed periodicity, trend components with continuous trend characteristics, and random components with sudden fluctuations, achieving precise splitting of time-series data with different attributes. The seasonal period of the STL decomposition is determined by the period extracted by the aforementioned FFT, with a default 24-hour daily period. The local smoothing window is set to 7 data points. The seasonal components directly reuse the load data from the same period in history, preserving the inherent periodic operation pattern of the server load. Meanwhile, the trend components with stable trends and strong continuity are input into the LSTM (Long Short-Term Memory) network model. Relying on the model's excellent time-series memory and fitting capabilities, the model accurately captures the long-term trend of load changes and outputs high-precision trend prediction values. The LSTM network is set to 2 hidden layers, each containing 128 neurons. The activation function is ReLU, the training iterations are 100, the batch size is set to 32, and a 5-fold cross-validation method is used during the model training phase to avoid overfitting.

[0047] The random components, characterized by fluctuations and sudden bursts, are input into the XGBoost extreme gradient boosting model. Leveraging the model's advantage in fitting nonlinear burst data, the prediction bias caused by sudden load fluctuations is corrected, and random prediction values ​​are output. The XGBoost model is configured with a tree depth of 6, a learning rate of 0.1, 50 iterations, and regularization parameters λ=0.1 and γ=0.01. Furthermore, the trend prediction results of LSTM are used as one of the input features of XGBoost to improve the prediction accuracy of random components.

[0048] After each component is predicted independently, the seasonal raw components, trend prediction values ​​and random prediction values ​​are integrated using an equal-weighted fusion method to generate the final overall server load prediction data.

[0049] Meanwhile, the model is equipped with a complete accuracy verification and iterative optimization mechanism. When the predicted MAPE exceeds 10%, the model is automatically fine-tuned online, updating the model parameters using the real load data of the most recent hour. The fine-tuning iteration count is 20 times. If the accuracy is still not up to standard after fine-tuning, the model is automatically switched to the ARIMA backup time series prediction model. The model completes full iterative training based on the latest cluster load data every day. The prediction accuracy is evaluated in real time by the mean absolute percentage error (MAPE). When the MAPE value exceeds the preset threshold of 10%, the model is automatically fine-tuned online, using the recent real load data to iteratively optimize the model parameters. If the accuracy is still not up to standard after fine-tuning, the model is automatically switched to the ARIMA backup time series prediction model to ensure the stability and accuracy of load prediction in extreme scenarios. The model also completes full iterative training based on the latest cluster load data every day to continuously adapt to the dynamic changes in cluster load patterns.

[0050] Further, step S03 above, the step of calculating the load fit of the MDS set based on the directory adjustment request using the load feature vector, the historical load profile, and the predicted load, includes: Step S031: parse the directory adjustment request, determine the directory type and directory access mode of the target directory, and obtain the directory characteristics based on the directory type and the directory access mode; Step S032: Calculate the real-time load score of several MDSs in the MDS set based on the load feature vector. Step S033: Calculate the historical fit score between the directory features and the several MDSs based on the historical load profile; Step S034: Calculate the predicted load score of the plurality of MDSs based on the predicted load; Step S035: Calculate the matching score between the directory access pattern and the pattern of the MDS based on the directory features and the hardware characteristics of the MDS. Step S036: The real-time load score, the historical fit score, the predicted load score, and the pattern matching score are weighted and summed to obtain the load fit.

[0051] After the cluster detects a directory adjustment request, it first parses the directory metadata information carried in the request, and combines it with the cluster's historical access logs to accurately distinguish whether the target directory belongs to a shared directory, a private directory, or a temporary directory. At the same time, it identifies the core access patterns of the directory, covering four mainstream patterns: random read intensive, sequential write intensive, mixed IO, and low-frequency access. Finally, it integrates these patterns to obtain a complete target directory feature dataset.

[0052] Based on the real-time load feature vectors of each metadata server, combined with the reverse indicator evaluation rules, the remaining capacity of the server under current load is quantitatively calculated, a real-time load score is generated, and server nodes with high load and resource saturation are avoided.

[0053] The extracted directory features are matched and compared with the historical load profiles of each server. Scoring is completed based on the adaptation rules of profile type and directory access mode. For example, a random read-intensive directory can get a high score when matched with a server with stable load, and a low score when matched with a server with drastic load fluctuation. This generates a historical adaptation score to ensure the fit between the long-term operation mode of the directory and the load characteristics of the server.

[0054] Based on the projected load data for each server, we predict the subsequent load growth trend and resource saturation risk, quantify and generate a projected load score, and proactively avoid server nodes that are about to be overloaded in the short term. Combining the hardware configuration characteristics, cache resource size, and cluster communication capabilities of each server with the access requirements and operating characteristics of the target directory, we assess the compatibility between hardware resources and directory access patterns, generating a pattern matching score to ensure that hardware resources accurately adapt to the directory service operation requirements.

[0055] Finally, according to the preset weighting rules, the current load and the predicted load are calculated using an inverse index rule, that is, the corresponding score is obtained by subtracting the standardized load value from 1. The lower the load, the higher the score. The basic weights of the four dimensions are as follows: real-time load weight 0.3, historical adaptability weight 0.25, predicted load weight 0.3, and pattern matching weight 0.15. The real-time load score, historical adaptability score, predicted load score, and pattern matching score are weighted and summed. The weights of each dimension can be dynamically fine-tuned according to the real-time running status of the cluster, with a single weight adjustment range not exceeding ±0.05. Finally, a load adaptability score that can comprehensively and multidimensionally quantify the node adaptability capability is generated, providing accurate data basis for directory binding decisions.

[0056] This embodiment, through the above scheme, specifically collects real-time load data of the metadata server MDS set, generates a load feature vector based on the load data; collects historical load data of the MDS set, constructs a historical load profile based on the historical load data, and performs load prediction on the historical load data using a combined prediction model to obtain the predicted load; when a directory adjustment request is detected, the load fit of the MDS set is calculated based on the directory adjustment request, using the load feature vector, the historical load profile, and the predicted load; and binds the target directory to the target MDS with the highest load fit in the MDS set according to the directory adjustment request. Therefore, by collecting multi-dimensional load data to generate load feature vectors, a fine perception of the current state of the MDS is achieved. Secondly, historical load profiles are constructed and predicted loads are obtained using a combined prediction model, giving the system forward-looking prediction capabilities and avoiding response lags in post-migration. On this basis, the load fit is calculated by integrating the current load, historical profiles, and predicted loads to achieve multi-dimensional fusion evaluation. Finally, the MDS with the highest fit is selected through intelligent pin decision-making, and the binding relationship between the directory and the MDS is adjusted to achieve intelligent dynamic allocation of directory metadata service nodes, rather than simple directory migration. At the same time, when dynamically adjusting the directory binding relationship at runtime, the risk of IO interruption is minimized by using the obtained load feature vectors, historical load profiles, and predicted loads, ensuring business continuity. This solves the problem in the existing CephFS system's MDS set load management where directory allocation lacks multi-dimensional load perception and forward-looking prediction, making it difficult to achieve efficient dynamic load balancing, and improves the efficiency of load adjustment.

[0057] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 In step S04, which involves binding the target directory to the target MDS with the highest load adaptability in the MDS centralized set according to the directory adjustment request, the directory binding method based on the distributed system CephFS further includes steps S041 to S043: Step S041: Filter the MDS set according to the load adaptability to obtain the target MDS; Step S042: According to the directory adjustment request, bind the target directory to the attribute identifier of the target MDS, and migrate the metadata access of the target directory to the target MDS to obtain the binding result.

[0058] like Figure 4As shown, after calculating the load adaptability of each metadata server, a node screening mechanism is executed first. A preset adaptability threshold is used to initially select candidate nodes, selecting all metadata servers that meet the adaptability standard to form a candidate node set. This effectively eliminates inferior nodes with excessive load, extremely poor matching, or potential future overload risks. The initial selection threshold is set to adaptability ≥ 0.6. If the candidate node set is not empty, the server with the highest adaptability is selected as the final target MDS. If the candidate node set is empty, the server with the highest global adaptability is selected as the alternative node, and a cluster expansion alarm is triggered. Each round of decision-making fully records the three-dimensional load data, adaptability calculation results, and final selection results of each MDS, generating a decision log for subsequent model optimization.

[0059] If there are valid nodes in the candidate node set, the server with the highest load adaptability value is selected as the final target MDS. If there are no valid nodes in the candidate node set, the server with the best global adaptability is selected as the alternative node. At the same time, a cluster expansion alarm is automatically triggered to remind the operation and maintenance personnel that the cluster resources are insufficient, so as to avoid cluster performance bottlenecks in advance.

[0060] After the target server is determined, the CephFS native setxattr system call is used to configure a unique ceph.dir.pin extended attribute for the target directory, binding the directory to the Rank number of the target MDS, and realizing the unique association between the directory and the node. This process is the execution stage of intelligent pin decision-making. Through load assessment, the MDS with the highest adaptability is selected, and the binding relationship of the directory is adjusted to realize the intelligent allocation of directory metadata service nodes.

[0061] After the binding operation is completed, a cache update notification is sent synchronously to all clients in the cluster, forcibly refreshing the client's local MDS access mapping cache, ensuring that all subsequent metadata requests for this directory are directly directed to the target server, thus completing the full migration of metadata access permissions.

[0062] Finally, based on the directory binding results, the parent directory, frequently accessed subdirectories, and core inode metadata of the target directory are actively retrieved and cached in batches in the target server's mdcache cache. This completes full cache warm-up, significantly reducing the latency of the first directory access and ensuring the access performance and stability of the business after migration. Specifically, after step S04 above, which involves binding the target directory to the target MDS with the highest load balancing performance in the MDS centralized dataset based on the directory adjustment request, the method further includes: Step S0401: When the migration triggering condition is met, the metadata of the target directory is copied to the alternative MDS for preheating to obtain the preheating result. Step S0402: Update the binding attribute identifier of the target directory to the alternative MDS, and notify the client to switch the access target to obtain the switching result; Step S0403: Verify the directory access status of the candidate MDS and obtain the verification result; Step S0404: If the verification result is a verification failure, the binding attribute identifier is restored to the original MDS, and the client is notified.

[0063] like Figure 5 As shown, after the directory completes the initial binding, the cluster operation status is continuously monitored. When any one of the four preset migration trigger conditions is met, the specific quantitative thresholds are: the target MDS current comprehensive load value ≥ 0.8, the target MDS predicted load in the next 5 minutes ≥ 0.7 and the trend is upward, the matching degree between the directory access mode and the current MDS profile after the change is < 0.5, and the cluster load balance is < 0.7. Among them, the load balance is the ratio of the minimum to the maximum adaptability of each MDS. The four-stage non-disruptive soft migration mechanism for business continuity is automatically started. This mechanism is different from the traditional subtree migration method. Through four stages, namely incremental warm-up, atomic switching, status verification and fault rollback, the risk of IO interruption during the migration process is minimized, and the business continuity of the directory adjustment process is guaranteed. The trigger conditions include the target server's current load exceeding the preset overload threshold, the target server's predicted load in the short term continuously rising and about to become overloaded, the directory access mode changing significantly, causing a significant decrease in the original node matching degree, and the overall cluster load balance being lower than the preset standard.

[0064] After the soft migration starts, it first enters the warm-up phase, which adopts an incremental metadata replication mechanism. Only the incremental metadata, inode information and lock status of the target directory changes are synchronized, without the need for full data replication. At the same time, a real-time metadata synchronization channel is established between the old and new nodes to asynchronously synchronize the real-time incremental change data of the directory and synchronously complete the cache warm-up of frequently accessed subdirectories. The entire process does not affect the normal access of existing services. The synchronization channel adopts an asynchronous IO mode, and the overall warm-up phase time is controlled within 100~300ms. The impact on the performance of existing services is controlled within 5%. This warm-up process realizes incremental synchronization of metadata, avoiding the huge overhead of traditional full quantum tree replication. At the same time, real-time synchronization through asynchronous IO ensures zero service interruption during the migration process.

[0065] After the preheating is complete, the atomic switchover phase begins. Cluster atomic operations are used to batch update the `ceph.dir.pin` binding attribute of the target directory, switching the directory binding relationship to the backup MDS. Simultaneously, the Ceph monitoring node broadcasts a switchover notification to all clients in the cluster, forcing clients to update their access targets. The original server stops receiving new access requests for that directory, ensuring no IO interruption during the switchover process. The switchover phase time is controlled within 50-100ms. Attribute updates are atomic operations, and global consistency of the broadcast notification is achieved through Ceph's Mon cluster. The original MDS synchronously stops receiving new access requests for that directory. In this embodiment, the atomic switchover process controls IO interruption time within 50-100ms, which is almost imperceptible to the client, significantly better than the hundreds of milliseconds to seconds of interruption caused by traditional migration locks.

[0066] After the switch is completed, the status verification phase begins. The directory access latency and access success rate of the alternative server are continuously monitored through a sliding window. At the same time, the consistency of metadata between the new and old nodes is verified by incremental hash comparison. The three verification standards are strictly controlled: access latency not exceeding 120% of the original latency, access success rate not less than 99.9%, and metadata hashes being completely consistent.

[0067] If all indicators meet the standards, the migration is considered successful, and the dynamic adjustment of the directory is officially completed. If any indicator fails to meet the standards or the verification fails, the automatic rollback mechanism is immediately activated. The Pin configuration snapshot saved before the switch is retrieved, the original binding attributes of the directory are restored with one click, and the client is notified to switch back to the target. The entire fault recovery process is automated. During the verification phase, a sliding window of 10 data points is used for latency monitoring. Incremental hash comparison is used for consistency verification to avoid the overhead of full verification. The verification phase takes 200~500ms. During the rollback phase, cluster resources are prioritized for execution, and the time is controlled within 50~100ms. Data recovery is completed based on the metadata snapshot saved before the switch to prevent business anomalies and ensure the continuity and stability of cluster services. Through incremental hash comparison and sliding window monitoring, data consistency is guaranteed and the performance overhead of full verification is avoided. Automatic rollback when verification fails further ensures the absolute continuity of business.

[0068] More specifically, after step S04 above, which involves binding the target directory to the target MDS with the highest load balancing performance in the MDS centralized dataset based on the directory adjustment request, the method further includes: Step S05: Collect the directory access latency, binding decision accuracy, and migration success rate of the MDS set; Step S06: Summarize the directory access latency, binding decision accuracy, and migration success rate to obtain server monitoring data; Step S07: When the preset adjustment period is reached, adjust the weight of the load adaptability index according to the server monitoring data.

[0069] Once the directory binding and dynamic migration process is complete, a full-time closed-loop monitoring mechanism is activated to continuously collect core operational and decision-making metrics from each metadata server within the cluster. The focus is on capturing core data such as average directory access latency, peak access latency, the accuracy of each round of binding decisions, the success rate of soft migration execution, and node load balancing, thereby achieving comprehensive awareness of the cluster's operational status.

[0070] All monitoring metrics data are aggregated according to a fixed statistical period. After removing abnormal monitoring data and completing data cleaning and organization, a standardized server cluster monitoring dataset is formed, which fully records the execution effect and running status of each round of directory binding, load migration, and node scheduling.

[0071] The system has a fixed adjustment cycle. After each cycle, based on the collected monitoring big data, it reviews the rationality of the weights of each dimension of load adaptability indicators, compares the weight ratio with the actual cluster operation effect, and accurately identifies weight deviation items.

[0072] Based on the deviation analysis results, the weights of various indicators in the adaptation calculation are dynamically optimized and adjusted. The magnitude of each weight adjustment is strictly controlled within ±0.05 to avoid decision-making deviations caused by large fluctuations in weights. At the same time, the parameters of the combined prediction model are periodically optimized and combined based on monitoring logs and real load data to continuously improve the accuracy of load prediction and decision-making. This forms a fully automated closed-loop optimization system of "data collection - status monitoring - review analysis - parameter optimization" to continuously adapt to the dynamically changing load scenarios and business access patterns of the cluster.

[0073] In this embodiment, the target MDS is obtained by filtering the MDS set according to the load adaptability as described above; according to the directory adjustment request, the target directory is bound to the attribute identifier of the target MDS, and the metadata access of the target directory is migrated to the target MDS to obtain the binding result. Therefore, by collecting multi-dimensional load data to generate load feature vectors, a fine perception of the current state of the MDS is achieved. Secondly, historical load profiles are constructed and predicted loads are obtained using a combined prediction model, giving the system forward-looking prediction capabilities and avoiding response lags in post-migration. On this basis, the load fit is calculated by integrating the current load, historical profiles, and predicted loads to achieve multi-dimensional fusion evaluation. Finally, the MDS with the highest fit is selected through intelligent pin decision-making, and the binding relationship between the directory and the MDS is adjusted to achieve intelligent dynamic allocation of directory metadata service nodes, rather than simple directory migration. At the same time, when dynamically adjusting the directory binding relationship at runtime, the risk of IO interruption is minimized by using the obtained load feature vectors, historical load profiles, and predicted loads, ensuring business continuity. This solves the problem in the existing CephFS system's MDS set load management where directory allocation lacks multi-dimensional load perception and forward-looking prediction, making it difficult to achieve efficient dynamic load balancing, and improves the efficiency of load adjustment.

[0074] For example, to help understand the implementation process of the directory binding method based on the distributed system CephFS obtained by combining this embodiment with the above embodiment one, please refer to... Figure 6 , Figure 6 A simplified flowchart illustrating a directory binding method based on the distributed system CephFS is provided, specifically: After the system starts, it continuously collects real-time multi-dimensional load data and time-series historical load data from each metadata server in parallel. The real-time load data is cleaned, standardized, and weighted and fused to generate a real-time load feature vector. At the same time, the historical load data is sharded and stored, and multi-dimensional features are extracted to build a unique historical load profile for each server. The load prediction is completed by using the STL-LSTM-XGBoost combined prediction model to output accurate future load prediction data.

[0075] The cluster monitors business requests such as directory creation and adjustment in real time. Once a valid request is detected, it immediately parses the type and access mode of the target directory, extracts the unique directory features, and combines the acquired real-time load features, historical load profiles, and predicted load data to quantitatively calculate the load adaptability of each server from four core dimensions, thus completing the selection of the optimal target MDS.

[0076] After the filtering is completed, the initial directory binding operation is performed. The directory is associated with the optimal node by configuring exclusive extended attributes, metadata access permissions are migrated synchronously and cache preheating is completed to achieve initial load balancing.

[0077] During cluster operation, the load status and business access status are continuously monitored. When the preset migration trigger conditions are met, the four-stage uninterrupted soft migration process is automatically started. Through incremental warm-up, atomic switching, status verification, and fault rollback, the dynamic optimization and adjustment of directory binding relationships are completed, ensuring that there is no IO interruption and no data anomaly throughout the process.

[0078] Ultimately, this solution achieves the core objectives of precise load awareness, intelligent pin decision-making, uninterrupted soft migration, and fully automated closed-loop optimization for CephFS multi-MDS clusters. It effectively addresses industry pain points such as uneven load distribution, migration interruptions, predictive delays, and high operational costs associated with traditional mechanisms. Specifically, intelligent pin decision-making achieves precise binding between directories and MDSs through three-dimensional load assessment, unlike traditional threshold-based post-migration. Uninterrupted soft migration reduces IO interruptions to millisecond levels through a four-stage mechanism, ensuring business continuity. Actual cluster testing has verified that this solution achieves the following technical effects: 80% reduction in hotspot MDS occurrence rate; cluster load balancing exceeding 0.8; 30% increase in metadata concurrent IOPS; metadata adjustment interruption duration less than 100ms; metadata consistency exceeding 99%; directory access success rate no less than 99.99%; mean absolute percentage error (MAPE) of load prediction ≤ 8%, a 46% improvement in accuracy compared to traditional prediction schemes; MDS load level classification accuracy no less than 95%; 30% improvement in mdcache cache utilization; 75% reduction in cache overload risk; and 50% reduction in operational costs through fully automated scheduling. It is also compatible with Ceph. Pacific and above versions are adapted for cloud-native dynamic directory scenarios. Incremental synchronization and dynamic rate limiting mechanisms ensure that the performance impact of migration and preheating on existing businesses does not exceed 5%, and the system resource ratio of the prediction module is less than 1%.

[0079] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the directory binding method based on the distributed system CephFS in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0080] This application also provides a directory binding device based on the distributed system CephFS, please refer to... Figure 7 The directory binding device based on the distributed system CephFS is applied to CephFS, and the directory binding device based on the distributed system CephFS includes: The acquisition module 10 is used to acquire real-time load data of the metadata server MDS set and generate a load feature vector based on the load data. Construction module 20 is used to collect historical load data of the MDS set, construct a historical load profile based on the historical load data, and perform load prediction on the historical load data through a combined prediction model to obtain the predicted load. The calculation module 30 is used to calculate the load adaptability of the MDS set based on the load feature vector, the historical load profile and the predicted load when a directory adjustment request is detected. Migration module 40 is used to bind the target directory to the target MDS with the highest load adaptability in the MDS collection according to the directory adjustment request.

[0081] The directory binding device based on the CephFS distributed system provided in this application, employing the directory binding method based on the CephFS distributed system described in the above embodiments, can solve the technical problem in the load management of MDS sets in existing CephFS systems where directory allocation lacks multi-dimensional load perception and forward-looking prediction, making it difficult to achieve efficient dynamic load balancing. Compared with the prior art, the beneficial effects of the directory binding device based on the CephFS distributed system provided in this application are the same as those of the directory binding method based on the CephFS distributed system provided in the above embodiments, and other technical features in the directory binding device based on the CephFS distributed system are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0082] This application provides a directory binding device based on the distributed system CephFS. The directory binding device based on the distributed system CephFS includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the directory binding method based on the distributed system CephFS in the above embodiment 1.

[0083] The following is for reference. Figure 8 This document illustrates a structural diagram of a directory binding device based on the distributed system CephFS, suitable for implementing embodiments of this application. The directory binding device based on the distributed system CephFS in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The directory binding device based on the distributed system CephFS shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0084] like Figure 8 As shown, a directory binding device based on the distributed system CephFS may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in read-only memory 1002 or a program loaded from storage device 1003 into random access memory 1004. Random access memory 1004 also stores various programs and data required for the operation of the directory binding device based on the distributed system CephFS. The processing unit 1001, read-only memory 1002, and random access memory 1004 are interconnected via bus 1005. Input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the directory binding device based on the distributed system CephFS to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows directory binding devices based on the distributed system CephFS with various systems, it should be understood that implementation or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0085] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0086] The directory binding device based on the CephFS distributed system provided in this application, employing the directory binding method based on the CephFS distributed system described in the above embodiments, can solve the technical problem in the load management of MDS sets in existing CephFS systems where directory allocation lacks multi-dimensional load perception and forward-looking prediction, making it difficult to achieve efficient dynamic load balancing. Compared with the prior art, the beneficial effects of the directory binding device based on the CephFS distributed system provided in this application are the same as those of the directory binding method based on the CephFS distributed system provided in the above embodiments, and other technical features in this directory binding device based on the CephFS distributed system are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0087] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0088] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0089] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the directory binding method based on the distributed system CephFS in the above embodiments.

[0090] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0091] The aforementioned computer-readable storage medium may be included in a directory-binding device based on the distributed system CephFS; or it may exist independently and not assembled into a directory-binding device based on the distributed system CephFS.

[0092] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by a directory binding device based on the distributed system CephFS, cause the directory binding device based on the distributed system CephFS to: collect real-time load data of the metadata server MDS set and generate a load feature vector based on the load data; Historical load data of the MDS set is collected, a historical load profile is constructed based on the historical load data, and a load prediction is performed on the historical load data using a combined prediction model to obtain the predicted load. When a directory adjustment request is detected, the load fit of the MDS set is calculated based on the directory adjustment request, using the load feature vector, the historical load profile, and the predicted load. The target directory is bound to the target MDS with the highest load fit in the MDS set according to the directory adjustment request.

[0093] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0095] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0096] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the directory binding method based on the distributed system CephFS described above. This solves the technical problem in existing CephFS systems where MDS set load management lacks multi-dimensional load awareness and forward-looking prediction in directory allocation, making it difficult to achieve efficient dynamic load balancing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the directory binding method based on the distributed system CephFS provided in the above embodiments, and will not be repeated here.

[0097] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the directory binding method based on the distributed system CephFS as described above.

[0098] The computer program product provided in this application can solve the technical problem in the load management of MDS sets in the existing CephFS system, where directory allocation lacks multi-dimensional load perception and forward-looking prediction, making it difficult to achieve efficient dynamic load balancing. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the directory binding method based on the distributed system CephFS provided in the above embodiments, and will not be repeated here.

[0099] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A directory binding method based on the distributed system CephFS, characterized in that, The directory binding method based on the distributed system CephFS is applied to CephFS, and the directory binding method based on the distributed system CephFS includes: Collect real-time load data from the metadata server MDS set, and generate a load feature vector based on the load data; Historical load data of the MDS set is collected, a historical load profile is constructed based on the historical load data, and load prediction is performed on the historical load data through a combined prediction model to obtain the predicted load. When a directory adjustment request is detected, the load fit of the MDS set is calculated based on the directory adjustment request using the load feature vector, the historical load profile, and the predicted load. Based on the directory adjustment request, the target directory is bound to the target MDS with the highest load adaptability in the MDS collection.

2. The directory binding method based on the distributed system CephFS as described in claim 1, characterized in that, The step of constructing a historical load profile based on the historical load data includes: The historical load data is stored according to a sliding window of preset time length to obtain window data; Perform time-domain statistical analysis on the window data to obtain time-domain characteristics; Periodic characteristics were obtained by performing periodic analysis on the window data using spectral analysis. The trend characteristics of the window data are determined by difference analysis and regression analysis. The historical load profile corresponding to the MDS set is constructed based on the time-domain features, the periodic features, and the trend features.

3. The directory binding method based on the distributed system CephFS as described in claim 1, characterized in that, The combined prediction model includes a first prediction model and a second prediction model. The step of using the combined prediction model to predict the load from the historical load data to obtain the predicted load includes: The historical load data is decomposed into seasonal, trend, and random components. The trend component is input into the first prediction model for prediction to obtain the trend prediction value; The random component is input into the second prediction model for prediction to obtain a random predicted value; The predicted load is obtained by fusing the seasonal component, the trend prediction value, and the random prediction value.

4. The directory binding method based on the distributed system CephFS as described in claim 1, characterized in that, The step of calculating the load fit of the MDS set based on the directory adjustment request, using the load feature vector, the historical load profile, and the predicted load, includes: The directory adjustment request is parsed to determine the directory type and access mode of the target directory, and the directory characteristics are obtained based on the directory type and access mode. Calculate the real-time load score of several MDSs in the MDS set based on the load feature vector; Calculate the historical fit score between the directory features and the several MDSs based on the historical load profile; Calculate the predicted load score of the plurality of MDSs based on the predicted load; The matching score between the directory access pattern and the several MDSs is calculated based on the directory features and the hardware characteristics of the several MDSs. The load fit is obtained by weighted summing of the real-time load score, the historical fit score, the predicted load score, and the pattern matching score.

5. The directory binding method based on the distributed system CephFS as described in claim 1, characterized in that, The step of binding the target directory to the target MDS with the highest load balancing degree in the MDS centralization according to the directory adjustment request includes: The MDS set is filtered according to the load adaptability to obtain the target MDS; Based on the directory adjustment request, the target directory is bound to the attribute identifier of the target MDS, and the metadata access of the target directory is migrated to the target MDS to obtain the binding result.

6. The directory binding method based on the distributed system CephFS as described in claim 1, characterized in that, After the step of binding the target directory to the target MDS with the highest load balancing in the MDS centralization according to the directory adjustment request, the method further includes: When the migration triggering conditions are met, the metadata of the target directory is copied to the alternative MDS for preheating to obtain the preheating result; Update the binding attribute identifier of the target directory to the alternative MDS, and notify the client to switch the access target to obtain the switching result; Verify the directory access status of the candidate MDS and obtain the verification results; If the verification result is a verification failure, the binding attribute identifier is restored to the original MDS, and the client is notified.

7. The directory binding method based on the distributed system CephFS as described in claim 1, characterized in that, After the step of binding the target directory to the target MDS with the highest load balancing in the MDS centralization according to the directory adjustment request, the method further includes: The directory access latency, binding decision accuracy, and migration success rate of the MDS set were collected. The server monitoring data is obtained by summarizing the directory access latency, binding decision accuracy, and migration success rate. When the preset adjustment period is reached, the weight of the load adaptability index is adjusted according to the server monitoring data.

8. A directory binding device based on the distributed system CephFS, characterized in that, The directory binding device based on the distributed system CephFS is applied to CephFS, and the directory binding device based on the distributed system CephFS includes: The acquisition module is used to acquire real-time load data of the metadata server MDS set and generate a load feature vector based on the load data. The construction module is used to collect historical load data of the MDS set, construct a historical load profile based on the historical load data, and perform load prediction on the historical load data through a combined prediction model to obtain the predicted load. The calculation module is used to calculate the load adaptability of the MDS set based on the load feature vector, the historical load profile, and the predicted load when a directory adjustment request is detected. The migration module is used to bind the target directory to the target MDS with the highest load adaptability in the MDS collection according to the directory adjustment request.

9. A directory binding device based on the distributed system CephFS, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the directory binding method based on the distributed system CephFS as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the directory binding method based on the distributed system CephFS as described in any one of claims 1 to 7.

Images